question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Could not add AVRO schema using producer client

See original GitHub issue

Describe the bug Unable to add AVRO schema on a topic using producer client.

To Reproduce Steps to reproduce the behavior:

  1. create a topic
  2. produce a message with json schema
Producer<User> producer = client.newProducer(Schema.JSON(User.class)).topic(Commons.TOPIC).create();
      User user = new User("rahul",31);
      MessageId msgId = producer.send(user);
  1. get schema api - return json schema type
{
    "getSchemaResponses": [
        {
            "version": 0,
            "type": "JSON",
            "timestamp": 0,
            "data": "{\"type\":\"record\",\"name\":\"User\",\"namespace\":\"com.optum.link.pulsar\",\"fields\":[{\"name\":\"name\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"age\",\"type\":\"int\"}]}",
            "properties": {
                "__jsr310ConversionEnabled": "false",
                "__alwaysAllowNull": "true"
            }
        }
    ]
}
  1. produce 2nd message with avro schema
Producer<User> producer = client.newProducer(Schema.AVRO(User.class)).topic(Commons.TOPIC).create();
      User user = new User("josh",31);
      MessageId msgId = producer.send(user);
  1. get schema api - returns only previously created json schema object. it did not include AVRO schema
{
    "getSchemaResponses": [
        {
            "version": 0,
            "type": "JSON",
            "timestamp": 0,
            "data": "{\"type\":\"record\",\"name\":\"User\",\"namespace\":\"com.optum.link.pulsar\",\"fields\":[{\"name\":\"name\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"age\",\"type\":\"int\"}]}",
            "properties": {
                "__jsr310ConversionEnabled": "false",
                "__alwaysAllowNull": "true"
            }
        }
    ]
}

Expected behavior Avro schema should be added pulsar schema registry, or if this is not feasible then producer client should throw error

Screenshots NA

Desktop (please complete the following information):

  • OS: iOS
  • Pulsar: 2.7.0
  • Pulsar Client: 2.7.0

Additional context I could successfully produce/consume messages using json schema.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
limingnihaocommented, Mar 4, 2021

I will fix this issue, please assign it to me. A check for Scheme Type will be added.

0reactions
codelipenghuicommented, Mar 2, 2021

@ta1meng We are working on the PR to fix this problem. The difference between the AVRO schema and JSON schema is the serialization, JSON schema encode to the JSON bytes and the AVRO schema encode to the AVRO format bytes.

Read more comments on GitHub >

github_iconTop Results From Across the Web

HowTo: Produce Avro Messages to Kafka using Schema ...
Here, I will show you how to send avro messages from the client application and from Kafka Streams using Kafka Schema Registry.
Read more >
Avro Schema Serializer and Deserializer
This document describes how to use Avro schemas with the Apache Kafka® Java client and console tools. Avro Serializer¶. You can plug KafkaAvroSerializer...
Read more >
Getting error while publishing message to kafka topic
I have written a simple JAVA program to generate a message using avro schema. I have generated a specific record. The record is...
Read more >
Using Apache Kafka with Schema Registry and Avro - Quarkus
This guide shows how your Quarkus application can use Apache Kafka, Avro serialized records, and connect to a schema registry (such as the...
Read more >
Kafka, Avro Serialization, and the Schema Registry - DZone
When using the Confluent Schema Registry, producers don't have to send schema — just the ... You can add a field with a...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found