Support PostgreSQL's JSON data type(s)

Original issue description:

Hi there,

in order to insert JSON data into the database, CrateDB’s documentation says:

You can insert objects using JSON strings. To do this, you must type cast the string to an object with an implicit cast (i.e., passing a string into an object column) or an explicit cast (i.e., using the ::object syntax). 1

Relating to 2, I wonder if CrateDB could provide a ::json alias to its type casting implementation underpinning the ::object syntax notation in order to implement at least some features of the official PostgreSQL JSON support [3,4], maybe even without further ado? In the case of 2, it might improve the compatibility with how Confluent’s JDBC Sink Connector for Kafka works 5.

@asavran reports at 2:

We tried to execute the same INSERT command manually by substituting the ::json with ::object and it worked. Is there a workaround for this?

I know that things might turn out to be more complicated more often than not, so I am just creating this as a basis for discussions.

With kind regards, Andreas.

Updated description:

---

Problem description

Kafka-connect-JDBC adds explicit ::json casts for target columns of type object (exposed in pg_catalog as json).

This currently breaks because json isn’t available as type.

The reason for kafka-connect-JDBC doing that is that otherwise PostgreSQL errors when using prepared statements. (http://sqlfiddle.com/#!17/7e29e/12, https://github.com/pgjdbc/pgjdbc/issues/265, https://github.com/confluentinc/kafka-connect-jdbc/pull/875)

Proposed potential solutions so far

1. Add a json alias for object
2. Add a dedicated new json type (without storage and cast support from text → json, json -> object, object → json)
3. Make kafka-connect-jdbc CrateDB aware or configurable, so that no explicit casts are added for json columns (see https://github.com/crate/crate/issues/11347#issuecomment-839751709)

Estimate is for option 2, which is the first choice here.

[hammerhead]

The issue can be reproduced by following our Kafka Connect Guide and adjusting the there mentioned cratedb_connector.json as follows:

{
  "name": "cratedb-connector",
  "config": {
    "connector.class": "io.confluent.connect.jdbc.JdbcSinkConnector",
    "topics": "metrics",
    "connection.url": "jdbc:postgresql://localhost:5432/doc?user=crate",
    "tasks.max": 3,
    "insert.mode": "insert",
    "table.name.format": "metrics",
    "transforms": "HoistPayload",
    "transforms.HoistPayload.field": "payload",
    "value.converter": "org.apache.kafka.connect.storage.StringConverter",
    "transforms.HoistPayload.type": "org.apache.kafka.connect.transforms.HoistField$Value"
  }
}

Confluent Connect derives the necessity to perform an explicit cast based on the column definition, see their PostgreSqlDatabaseDialect.java.

PG has a different syntax to interact with json fields, though we already return a pg json (oid 114) on SELECT on an OBJECT anyway through jdbc

Returning the column as json in pg_catalog is exactly what causes Confluent Connect to add the ::json cast. In this commit the cast was added:

the PostgreSqlDatabaseDialect class is modified to add the cast expression (e.g., ::uuid) to the variable placeholder (e.g, ?) if the column type matches one of the database types that requires casting, which in this case is either uuid, json, or jsonb.

In my test cases on PostgreSQL (12.6), inserts into JSON/JSONB columns worked fine without explicit casting, but I guess Confluent makes the assumption that if a column signals to be a json in pg_catalog, it would also support inserting values casted to json. Maybe it was required in older PostgreSQL versions to have that explicit cast.

[hammerhead]

FYI, I was able to successfully test the new JSON data type with the Confluent JDBC Sink connector. An updated version of the Kafka Connect Howto is available as a draft for when 4.7 is released (crate/crate-howtos#290).