Optimization: valuesAtQuantiles qdigest functions

I noticed that these quantile functions on a qdigest compute the percentile values by looping over the provided percentiles array argument and calling the underlying airlift qdigest method getQuantile (singular).

However, the airlift qdigest object has a plural version of this method getQuantiles which seems to be more efficient because it only traverses the qdigest tree once for a given array of quantiles.

I haven’t had a chance to quantify this optimisation yet, but wanted to gauge whether there’d be any opposition to a pull request on this topic?

Thanks

[blrnw3]

I did some perf testing on this and interestingly found no measurable difference in the two approaches. I tested on the TPCH data on several scale factors, evaluating 20 percentiles with the two approaches outlined in the issue summary. It seems the qdigest building is so dominant that extracting the percentiles has trivial cost, regardless of how it is done.

[tdcmeehan]

I agree with @mehrdad-honarkhah, and we’ll be happy to review a pull request for this optimization.

Please be aware, the plural version of the function validates that the quantiles are sorted in ascending order. To preserve compatibility with approx_percentile and existing users of values_at_quantiles, we don’t want to introduce that validation to the values_at_quantiles function. I think it would also be worthwhile to port this over to approx_percentile, as it does the same thing currently.