Can I read DataColumnStatistics of the column only before reading the entire column data ?

Hi

Is there a way to read only the DataColumnStatistics before actually loading the entire column data into memory ? Essentially I have a method that checks if the search value exists in the data column by checking column Min and Max value

public static bool ValueExistsInColumnRange<T>(this DataColumn column, T value)
        where T : IComparable<T>
    {
        if (value is null || column.Statistics.MinValue is null || column.Statistics.MaxValue is null ||
            ((T)column.Statistics.MinValue).CompareTo(value) > 0 || ((T)column.Statistics.MaxValue).CompareTo(value) < 0)
            return false;

        return true;
    }

and if value doesn’t happen to exist in within the column then I can skip entire column without loading its vaules into memory. However I’ve noticed that ParquetRowGroupReader.ReadColumnAsync loads the entire column data into memory. How to only load column statistics and optionaly load column data on demand ?

[mirosuav]

@aloneguid no, I don’t know GeoParquet 😃 We’re doing our own research on comparing different storage format for big real time data.

[mirosuav]

Thanks, @aloneguid. I already have a working solution for that, will PR it once I’m done testing.