Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Proposal to form a sub-working group for a schema registry API

See original GitHub issue

Microsoft proposes the formation of a sub-working group to starts working on a common schema registry API interface in parallel with the ongoing efforts.

The schema registry API should, in our view, fulfill the following minimal requirements:

  • Provide a management and retrieval interface over HTTP (AMQP services can leverage the HTTP-over-AMQP binding that is pending in the AMQP TC)
  • Allow for storing schema file streams for any data serialization format for both schema-driven serialization and data validation, e.g. Avro or Protobuf or XML or JSON schemas.
  • Allow for storing for multiple versions for each schema.
  • Allow for storing of logical groups of such schemas.

Our concrete initial proposal is to create a very simple REST API for this purpose, that manages a resource graph like this:**schemagroups**/{schemagroup}/**schemas**/{schema}/**versions**/{version}

  • “schemagroups” collection: the top level collection of schema groups. Each group is a logically related set of schemas, typically belonging to a particular application or subject to shared access control rules.
  • “schemas” collection: the collection of schemas belonging to a schema group.
  • “versions”: the versions of schema documents belonging to a particular schema.

The version identifiers, eg. .../versions/4 are assigned and managed by the service and do not necessarily reflect the developer-assigned version number. A service implementation MAY enforce certain dependencies between versions. For instance, a service implementation that specifically understand Avro schemas might be configured such that a new schema version can only be added if it satisfies certain backwards compatibility criteria. If the added schema does not satisfy those criteria, the service might then report a conflict (409).

In the “versions” collection, clients can add new schemas with POST against the collection root. The response is either 201 (Created) with a Location header that indicates the newly created schema version location, 400 (Bad Request) for a malformed request, 409 (Conflict) for a consistency violation as just discussed, or some other 5xx or 4xx response appropriate for the error condition.

Schema version can be deleted but not replaced, meaning there is no update.

The “versions” collection allows enumerating all versions of a schema with GET. The result is a JSON array holding URI-references to to the schema version documents. Schema version documents can be deleted with DELET on their schema version URI.

In the “schemas” collection clients can add new schemas by name using the PUT method with the schema name in the URI segment. This creates a new version collection for the schema if required and then appends the schema as the latest version. A GET against “…/schemas/{schema}” will return the latest version of the schema document stored in the schema collection.

A schema version collection including all stored schemas is deleted with a DELETE on “…/schemas/{schema}”.

The management of schema groups is equivalently straightforward. A PUT on “/schemagroups/{schemagroup}” with the group name in the segment creates the group. A DELETE deletes it.

Authorization is done using existing HTTP mechanisms, e.g. OAuth Bearer tokens.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:12
  • Comments:13 (8 by maintainers)

github_iconTop GitHub Comments

duglincommented, May 9, 2020

This was approved on the 5/7 call. I’ll work with @clemensv off-line on the next steps in the process

erikeriksoncommented, Jul 22, 2021

This seems bypassed by actions. I support closing it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Schema Registry API Usage Examples
This section provides examples of calls to the Schema Registry API using curl ... Start Schema Registry and its dependent services ZooKeeper and...
Read more >
Getting Started with the Schema Registry API
This document provides an introduction to the core concepts you need to know before attempting to make calls to the Schema Registry API....
Read more >
Event Streams schema registry - IBM Cloud Docs
Each Enterprise plan Event Streams instance has its own schema registry. ... When you create schemas by using the REST API, schemas are...
Read more >
Integrating with AWS Glue Schema Registry
Let's assume you are writing data to an Apache Kafka topic, and you can follow these steps to get started. Create an Amazon...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found