Low performance while using `latest` as `schema.id`See original GitHub issue
SchemaLoader uses unchached call to get the latest version id:
This happens quite often and results in a huge amount of http requests to schema registry. This value could be cached for some time period, and the time period should be configurable.
- Created 3 years ago
- Comments:15 (1 by maintainers)
Top GitHub Comments
Hi @agolovenko , my comments on your comments.
The method you’re referring to is
getLatestSchemaMetadata, right? This one
If yes, then of course it has to constantly query Schema Registry for the latest version, however, this only happens when getting the schema to be used by Catalyst, as you can browse here to confirm
The subsequent calls can and must be cached since they are based on the id available on top of the payload for each record, as you can see here
Anyway, thank you very much for coming back and we’ll soon release an improvement for this.
You are welcome. Since there seems to be no issue any more, I’m closing this ticket. Please open a new one if you have any problems.