Support ROW comparison for fields with NULL elements

The issue is manifested when we’re calling checksum on row-typed columns, and some rows contains null elements.

To reproduce:

SELECT array_sort(array_agg(row(x, y)))
FROM (VALUES (null, null), (1, 2)) AS t(x, y)

stacktrace:

com.facebook.presto.spi.PrestoException: ROW comparison not supported for fields with null elements
        at com.facebook.presto.spi.type.RowType.checkElementNotNull(RowType.java:269)
        at com.facebook.presto.spi.type.RowType.compareTo(RowType.java:240)
        at com.facebook.presto.operator.scalar.ArraySortFunction$1.compare(ArraySortFunction.java:83)
        at com.facebook.presto.operator.scalar.ArraySortFunction$1.compare(ArraySortFunction.java:66)
        at java.base/java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
        at java.base/java.util.TimSort.sort(TimSort.java:220)
        at java.base/java.util.Arrays.sort(Arrays.java:1440)
        at java.base/java.util.List.sort(List.java:506)
        at java.base/java.util.Collections.sort(Collections.java:176)
        at com.facebook.presto.operator.scalar.ArraySortFunction.sort(ArraySortFunction.java:65)
        at com.facebook.presto.$gen.PageProjectionWork_20190416_225748_08059_xkma5_0_7_570478.evaluate(Unknown Source)
        at com.facebook.presto.$gen.PageProjectionWork_20190416_225748_08059_xkma5_0_7_570478.process(Unknown Source)
        at com.facebook.presto.operator.project.DictionaryAwarePageProjection$DictionaryAwarePageProjectionWork.process(DictionaryAwarePageProjection.java:175)
        at com.facebook.presto.operator.project.PageProcessor$ProjectSelectedPositions.processBatch(PageProcessor.java:295)
        at com.facebook.presto.operator.project.PageProcessor$ProjectSelectedPositions.process(PageProcessor.java:183)
        at com.facebook.presto.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:315)
        at com.facebook.presto.operator.WorkProcessorUtils$YieldingIterator.computeNext(WorkProcessorUtils.java:79)
        at com.facebook.presto.operator.WorkProcessorUtils$YieldingIterator.computeNext(WorkProcessorUtils.java:65)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
        at com.facebook.presto.operator.project.MergingPageOutput.getOutput(MergingPageOutput.java:111)
        at com.facebook.presto.operator.FilterAndProjectOperator.getOutput(FilterAndProjectOperator.java:102)
        at com.facebook.presto.operator.Driver.processInternal(Driver.java:379)
        at com.facebook.presto.operator.Driver.lambda$processFor$8(Driver.java:283)
        at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:675)
        at com.facebook.presto.operator.Driver.processFor(Driver.java:276)
        at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1077)
        at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162)
        at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:483)
...

[highker]

I kinda remember we need is distinct from operator rather than = for null comparison.

BTW, that is a FB internal link lol.

[wenleix]

We should check SQL Standard about how to sort structural types with NULL values inside; e.g. does NULL LAST applies “recursively” into structural types?

I would close this issue for now since for the Verifier problem, we can cast them into JSON as workaround.