[Feature Request] Handling optional parts in date format

Hello,

I think It could be a nice feature to handle optional parts in date format with [] For example in my case, I can have DateTime with second fractions or not (it’s adaptive).

Actually, I need to perform a Substr :

FromDateTime(Substr(JSONPATH(payload, '$.created_at'), 0, 18), 'yyyy-MM-dd''T''HH:mm:ss')

But It could be handled like that :

FromDateTime(JSONPATH(payload, '$.created_at'), 'yyyy-MM-dd''T''HH:mm:ss[.SSS]''Z''')

It’s already provided by Java 8 and DateTimeFormatter, in documentation we have :

   [       optional section start
   ]       optional section end

[KKcorps]

The above benchmarks were for Java 11. For Java 17, they seem much closer. And converting String to Epoch is actually faster with java.time

Benchmark                                     Mode  Cnt     Score      Error  Units
BenchmarkJoda.benchmarkJavaTime               avgt    5  4986.148 ±  551.520  us/op
BenchmarkJoda.benchmarkJavaTimePreCompute     avgt    5  4075.269 ±  691.534  us/op
BenchmarkJoda.benchmarkJoda                   avgt    5  3719.916 ±  319.401  us/op


BenchmarkJoda.benchmarkJavaDateStringToEpoch  avgt    5  6362.567 ± 1251.276  us/op
BenchmarkJoda.benchmarkJodaStringToEpoch      avgt    5  9887.073 ± 1456.975  us/op

[richardstartin]

Case in point, I found that Joda was 6x faster than Java time for year conversion here: https://github.com/apache/pinot/pull/8397#issuecomment-1077589112

There has been some progress in java.time recently:

• https://github.com/openjdk/jdk/pull/7957
• https://github.com/openjdk/jdk/pull/7989
• https://github.com/openjdk/jdk/pull/7990
• https://github.com/openjdk/jdk/pull/8014
• https://github.com/openjdk/jdk/pull/8039

Some of these improvements will be backported to JDK17, so we should reassess once Pinot can run on JDK17.