Proposal to form a sub-working group for a schema registry API

Microsoft proposes the formation of a sub-working group to starts working on a common schema registry API interface in parallel with the ongoing efforts.

The schema registry API should, in our view, fulfill the following minimal requirements:

• Provide a management and retrieval interface over HTTP (AMQP services can leverage the HTTP-over-AMQP binding that is pending in the AMQP TC)
• Allow for storing schema file streams for any data serialization format for both schema-driven serialization and data validation, e.g. Avro or Protobuf or XML or JSON schemas.
• Allow for storing for multiple versions for each schema.
• Allow for storing of logical groups of such schemas.

Our concrete initial proposal is to create a very simple REST API for this purpose, that manages a resource graph like this:

https://example.com/**schemagroups**/{schemagroup}/**schemas**/{schema}/**versions**/{version}

• “schemagroups” collection: the top level collection of schema groups. Each group is a logically related set of schemas, typically belonging to a particular application or subject to shared access control rules.
• “schemas” collection: the collection of schemas belonging to a schema group.
• “versions”: the versions of schema documents belonging to a particular schema.

The version identifiers, eg. .../versions/4 are assigned and managed by the service and do not necessarily reflect the developer-assigned version number. A service implementation MAY enforce certain dependencies between versions. For instance, a service implementation that specifically understand Avro schemas might be configured such that a new schema version can only be added if it satisfies certain backwards compatibility criteria. If the added schema does not satisfy those criteria, the service might then report a conflict (409).

In the “versions” collection, clients can add new schemas with POST against the collection root. The response is either 201 (Created) with a Location header that indicates the newly created schema version location, 400 (Bad Request) for a malformed request, 409 (Conflict) for a consistency violation as just discussed, or some other 5xx or 4xx response appropriate for the error condition.

Schema version can be deleted but not replaced, meaning there is no update.

The “versions” collection allows enumerating all versions of a schema with GET. The result is a JSON array holding URI-references to to the schema version documents. Schema version documents can be deleted with DELET on their schema version URI.

In the “schemas” collection clients can add new schemas by name using the PUT method with the schema name in the URI segment. This creates a new version collection for the schema if required and then appends the schema as the latest version. A GET against “…/schemas/{schema}” will return the latest version of the schema document stored in the schema collection.

A schema version collection including all stored schemas is deleted with a DELETE on “…/schemas/{schema}”.

The management of schema groups is equivalently straightforward. A PUT on “/schemagroups/{schemagroup}” with the group name in the segment creates the group. A DELETE deletes it.

Authorization is done using existing HTTP mechanisms, e.g. OAuth Bearer tokens.

[duglin]

This was approved on the 5/7 call. I’ll work with @clemensv off-line on the next steps in the process

[erikerikson]

This seems bypassed by actions. I support closing it.