`computeTruncatedLength` may cause 20% table scan slow down for Raptor (or Hive)

I’m looking into raptor performance recently. It turns out computeTruncatedLength may eat 20% CPU (from my local benchmark). This function is a sanity check to make sure unicode codepoints are valid otherwise truncate. Though it is necessary, no sure if there is other way to avoid such high overhead.

Benchmark: https://github.com/highker/presto/commit/4b74a468f3b0d60799603a551e6d8fe0eb7b531b

Results:

without computeTruncatedLength: 569.7778528263376 MB/s with computeTruncatedLength: 441.1912459425512 MB/s

The table (orc format, single file, 7.5GB, 150M roles) I used for this benchmark is a materialized tpch table with a varchar column.

[dain]

Does the benchmark contain any data with multibyte characters? If not, I would expect the whole thing to generate assembly with just validation of the assumption and no real byte counting due to this check https://github.com/prestodb/presto/blob/master/presto-spi/src/main/java/com/facebook/presto/spi/type/Varchars.java#L83 . If that is not happening, I’d look into restructuring the code so that the inlining happens, or hoist that check closer to the main loop so that the common path avoids that call entirely.

[highker]

Aha~ Works very well~~~~~~