question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Convert from avro logical types to .net equivalents (and viceversa)

See original GitHub issue

Description

I’m trying to find out a way to use the avro logical types, behind the scene the avrogen tool creates a byte[] property (for decimals) and couldn’t find a way to convert the .net decimals into this.

Can you please help me with this?

How to reproduce

  1. Create a schema file, e.g. User.avsc

    [
      {
        "namespace": "Confluent.Kafka.Examples.AvroSpecific",
        "type": "record",
        "name": "User",
        "fields": [
          {
            "name": "name",
            "type": "string"
          },
          {
            "name": "favorite_number",
            "type": [ "int", "null" ]
          },
          {
            "name": "favorite_color",
            "type": [ "string", "null" ]
          },
          {
            "name": "birth_date_raw",
            "type": {
              "type": "int",
              "logicalType": "date"
            }
          },
          {
            "name": "salary_raw",
            "type": {
              "type": "bytes",
              "logicalType": "decimal",
              "precision": 19,
              "scale": 4
            }
          }
        ]
      }
    ]
    
  2. Run the avrogen tool in the console.

    avrogen.exe -s .\User.asvc .
    
  3. See the generated property in the User.cs (generated class)

    public byte[] salary_raw
    {
       get
       {
          return this._salary_raw;
       }
       set
       {
          this._salary_raw = value;
       }
    }
    

Checklist

Please provide the following information:

  • Confluent.Kafka nuget version:
  • Apache Kafka version:
  • Client configuration:
  • Operating system:
  • Provide logs (with “debug” : “…” as necessary in configuration)
  • Provide broker log excerpts
  • Critical issue

Issue Analytics

  • State:open
  • Created 5 years ago
  • Reactions:5
  • Comments:7 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
stefan-benjamincommented, Sep 27, 2019

Any news regarding this issue? I am myself new to Avro and would like to generate a C# model based on a Avro schema .asvc file using the avrogen tool. One of the properties defined in the schema has a ‘timestamp-milis’ logical type that I would like converted to a C# type.

https://avro.apache.org/docs/1.8.0/spec.html#Timestamp

1reaction
codeclashcommented, Jun 12, 2019

In case this might help: where s is the avro encoded decimal, and you know the original scale - using @manuel-zulian 's method:

public static decimal GetDecimalFromBase64String(string s, int scale)
{
   var bytes = Convert.FromBase64String(s);
   var result = new BigInteger(bytes.Reverse().ToArray());
   return (decimal)result * (decimal)Math.Pow(10, -scale);
}
Read more comments on GitHub >

github_iconTop Results From Across the Web

Avro C#: Avro.Util.LogicalType Class Reference
Converts a logical value to an instance of its base type. Parameters. logicalValue, The logical value to convert. schema, The schema that represents...
Read more >
How to hook a conversion for the logical type in Avro ...
I am looking for the same information regarding how to hook the conversion. Currently the conversion is always null and I am not...
Read more >
Type Conversion in .NET
Type conversion creates a value in a new type that is equivalent to the value of an old type, but does not necessarily...
Read more >
Kafka Connect | Confluent Documentation
Connectors and tasks are logical units of work and must be scheduled to ... For example, using the same Avro converter, the JDBC...
Read more >
PXF – Introducing support for reading the Avro Logical Types
A logical type is always serialized using its underlying Avro type so that values are encoded in exactly the same way as the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found