Re-design the date time format

Currently the date time format is of the format:

• <size>:<timeUnit>:<timeFormat>(:<pattern>( tz(<timezone>))

E.g.

• 1:SECONDS:EPOCH
• 1:DAYS:SIMPLE_DATE_FORMAT:yyyy-MM-dd tz(America/Chicago)
• 1:MILLISECONDS:TIMESTAMP

The problem of this format is that for certain time format, the size and time unit is implicit, and will cause confusion.

The proposal is to change the time format to:

• EPOCH|<timeUnit>(|<size>)
  • EPOCH|SECONDS
  • EPOCH|MINUTES|5
• SDF|<timeFormat>(|<timezone>)
  • SDF|yyyy-MM-dd
  • SDF|yyyy-MM-dd'T'HH:mm:ss'Z'|America/Chicago
• TIMESTAMP

We want to change the delimiter so that it won’t be mixed with the character in the time format

[Jackie-Jiang]

@KKcorps Agree that keeping it as JSON would be easier to use. We still need a single string representation though which can be used as the argument of the DATE_TIME_CONVERT function. The changes proposed here is for the existing DateTimeFieldSpec and DATE_TIME_CONVERT function, and we should do it in a backward compatible way. The long term solution should be to introduce the DATE_TIME data type, which can take extra properties (e.g. pattern, timeZone).

[KKcorps]

I feel both of these formats cause a lot of confusion for new users. What each value represents is not implicit and requires going through the doc to change every time. I propose a better way to handle this is to just keep this whole field as JSON. e.g.

{
      "granularity":1,
      "granularityTimeUnit":"DAYS",
      "inputTimeFormat":"SIMPLE_DATE_FORMAT",
      "pattern":"yyyy-MM-dd",
      "timeZone":"PST"
}