Consider removing `local_datetime`

Idea

Make a single datatype for a timestamp which is basically UTC datetime.

Motivation

1. There are very little use cases (if any) of the type
2. It’s a source of lots of errors
3. If you really want it, think twice and use two fields for date and time (i.e. date when alarm first started and time when it should ring for a recurring alarm, instead of cobining the two)

What other databases are doing?

Terms:

• naive – timestamp without timezone specified (for example 2019-11-25 22:10:42 if written as string)
• unix timestamp – specific representation of datetime in seconds since the unix epoch (epoch is in UTC so the whole value doesn’t depend on the time zone)
• UTC-normalized timestamp – means datetime stored as UTC internally being it unix timestamp or timestamp at UTC

Databases:

• PostgreSQL has TIMESTAMP which is documented to be UTC but accepts a naive timestamp as an input, so easy to misuse. And TIMESTAMPTZ which accepts any timezone offset as input and converts to UTC timestamp internally. Converts to client session time on output.
• MySQL has TIMESTAMP which accepts a naive timestamp in local time and does autoconversion to UTC for storage. DATETIME is just a naive timestamp without conversion
• VoltDB uses UTC-normalized TIMESTAMP. As long as I understand it has Java API and it’s hard to mess with timezones (because it’s converted from java.date.Date that is basically a unix timestamp)
• MongoDB has UTC-normalized timestamp named Date (as far as I understand when converting to/from a string they only accept Z-suffixed format 2019-11-25T22:10:42Z). Name probably inherits JavaScript type name
• Clickhouse use naive format that is converted to/from local time, named DateTime (basically like TIMESTAMP in mysql)
• FoundationDB (in document layer) basically supports MongoDB semantics
• RethinkDB actually store timestamp + timezone, named time
• Cassandra uses UTC-normalized timestamp and auto-converts to timezone in some clients
• Redis has no datatype for storing datetimes but uses unix timestamp (in seconds) in commands like EXPIREAT

Conclusions:

1. Except in PostgreSQL and MySQL every database has a single datetime type (most of them don’t have separate date or time, though)
2. Whereas PostgreSQL and MySQL looks like were trying to fix a mistake, that’s why they added a type, not because two types needed
3. Timestamp means different things in different databases so it’s okay to give it our own meaning (i.e. rename datetime -> timestamp)

Update: added cassandra

[1st1]

Alright, I’ve given this some thought in light of the discussion of this topic on our team call today. Here’s the TL;DR part:

• I’m -1 on removing the local_datetime datatype.
• I’m +1 to rename local_datetime, local_time, and local_date to clock_datetime, clock_time, and clock_date.

At this point I think we should wind down this discussion and focus on shipping Alpha 2.

---

Paul’s main motivation to remove the local_datetime type falls down to:

1. Users can be confused and pick the wrong datatype for storing their date/time data.
2. If the use case for the datatype is too marginal we should not have it. Having “nice to haves” increases maintenance costs and mental overhead.
3. Other databases do not have naive date/time type.
4. Some programming languages (including recent ones) don’t have naive date/time type.

Regarding (1): I believe this should be addressed by providing a better documentation and appropriate naming. I admit that the current “local” term can be confusing. Since in all of our discussions we always say that “local time” is about “clock time” I think we should just rename our naive types to use the word “clock”.

Regarding (2): First, I don’t think that the use case is marginal. We did use a “reminders” app as an example for when some developers would prefer to use local_datetime. But it goes deeper than that; notifications, schedules, do not disturb times, alarms: they all conceptually operate in local time to the user. I can totally see our own cloud offering having user settings for when we should send our users performance reports or analytics. Such settings are location independent, i.e. it does not matter where the user is, did a DST transition happen or not—a user just wants to see their report on Wednesday at 10am local time.

Paul argues that for a marginal use case like that a tuple of (local_date, local_time) can be used. True, but then there is no way our users can perform date/time arithmetic on such values. How would a user shift all their local date/time values 10 minutes to the future? Or how do you express “send me a message 10 minutes prior to [this local date/time]”? How would you implement school schedule app without using local date/time? Saying all these examples are “not realistic” or “not relevant” does not cut it, sorry. And here we arrive to another topic: how do we want to design EdgeDB. What features should make it and what should not?

In my opinion, it’s finding the balance between understanding if a feature addresses a marginal or a popular use case, and between whether it is possible to emulate the feature if a user is facing that marginal use case. And for local_datetime the answer is simple: implementing date/time arithmetic for a (local_date, local_time) tuple would move our users to the world of pain. They wouldn’t be able to use std::duration; they would either have to implement their custom date/time arithmetic via EdgeQL functions or to fetch the data and perform the calculation on the client side. But what if a user wants to update millions of values? All of them? This quickly becomes impractical.

If we want to be fanatical about this we can also drop std::local_date and std::local_time —they can also be easily implemented with a simple named tuple. And probably we don’t want std::duration at that point. Would that be convenient? Well, for some users it wouldn’t matter, but some would just continue using PostgreSQL because it offers all these features and EdgeDB would be just inferior to it.

We want EdgeDB and EdgeQL to be feature rich so that people solve less of data-related mundane problems on the client side. So let’s continue doing that.

Further, I don’t believe that maintaining std::local_datetime would require any significant effort.

Regarding (3): this is a rather weak point as many databases nowadays are treated as dumb data stores. Little effort, if any, goes into designing good standard libraries, data types, and query languages. Naturally our philosophy is quite different from theirs and this is what separates EdgeDB from the competition.

Regarding (4): this doesn’t really matter in this context. When you are dealing with a programming language it is easy to just install a third-party library for any missing functionality. Databases are different—it’s not easy to install an extension in your hosted DB. Building useful date/time arithmetic abstractions for (local_date, local_time) tuples would be problematic even in EdgeQL.

---

That said I’m grateful to Paul for highlighting the fact that our current naming is non-obvious. “local” can be ambiguous to users, and “naive” or “clock” prefix would read way better.

I think we should rename std::local_ types to std::naive_ or std::clock_. Either name would work fine IMO.

I’m -1 on renaming std::datetime to std::timestamp. The key problem is that the timestamp type in Postgres is a direct opposite to our std::datetime. Given that EdgeDB is based on Postgres I can imagine that a lot of users would be confused and would be looking for the std::timestamptz type spending time and asking questions on StackOverflow.

I’m -1 on renaming std::local_datetime to std::date. JavaScript and Swift use Date for full date/time types, so I think that std:date would be confusing.

[tailhook]

Decided to move it into a namespace #902