DISTINCTCOUNT and DISTINCTCOUNTHLL aggregations are really slow

Pinot version used is 0.4.0

Issue: DISTINCTCOUNT on a string column is much much slower than the group by on that same column. DISTINCTCOUNT query:

Select DISTINCTCOUNT(foo) FROM myTable WHERE col1 = 'value1' AND time_col >= 1596750994918 AND time_col < 1597960594918 AND foo != 'null' limit 1

group by query

Select foo, COUNT(*) FROM myTable WHERE col1 = 'value1' AND time_col >= 1596750994918 AND time_col < 1597960594918 AND foo != 'null' group by foo limit 10000

We don’t have an index on column foo. The DISTINCTCOUNT query was taking anywhere b/w 3 and 6s while the group by query returns in <500ms most of the time. DISTINCTCOUNTHLL was equally slower, if not more in some cases, than DISTINCTCOUNT query.

cc @kishoreg

[kishoreg]

one reason might be that group by was using dictionary whereas distinct is not. I think you have fixed this in the recent version.

[kishoreg]

@buchireddy please update this ticket once you have the new version