question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Return type definition of Kafka.structured seems incorrect

See original GitHub issue

Describe the Bug

The types for return values from Kafka.structured(cloudEventInstance) seem to be incorrect. The returned object has the following structure if viewed using logs or a debugger:

{
  key: 'fubar',
  value: '{"id":"ef233aa0-00a0-4b5a-9886-f8ff7566ef50","time":"2022-04-14T20:26:32.291Z","type":"foo","source":"bar","specversion":"1.0","partitionkey":"fubar","data":{"hello":"world}}',
  headers: { 'content-type': 'application/cloudevents+json; charset=utf-8' },
  body: '{"id":"ef233aa0-00a0-4b5a-9886-f8ff7566ef50","time":"2022-04-14T20:26:32.291Z","type":"foo","source":"bar","specversion":"1.0","partitionkey":"fubar","data":{"hello":"world}}',
  timestamp: '1649967992291'
}

However, the types come from CE.Message<string>. This results in the following intellisense structure being suggested:

{
  body: unknown,
  headers: CE.Headers
}

It seems as though the headers key is correct, but the body is incorrect, and other keys are missing.

Steps to Reproduce

  1. Create a CloudEvent, e.g new ce.CloudEvent({ type: 'foo', source: 'bar', data: { hello: 'world' }, partitionkey: 'foobar' }
  2. Pass it to, e.g Kafka.structured(theEvent)
  3. Attempt to access the timestamp, value, or key. These are not known properties on the return value.
  4. Try to use the body and you find it is incorrectly typed as unknown

Expected Behavior

I would have expected the types to reflect the Object in the Describe the bug section, e.g:

{
  key: string|undefined
  value: string,
  headers: CE.Headers,
  body: string,
  timestamp: string
}

Is this an invalid assumption to make? Passing the result of Kafka.structured(cloudEventInstance) object to a Kafka client such as KafkaJS requires casting the object to access the necessary properties to satisfy the TS compiler. It seems like this casting should be unnecessary.

Additional context

I’m using version 6.0.1 of the cloudevents module. The image below might help provide some context.

Screenshot 2022-04-14 at 1 45 23 PM

Issue Analytics

  • State:open
  • Created a year ago
  • Reactions:1
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
lancecommented, May 10, 2022

@FelixHarvey @evanshortiss thank you for your detailed descriptions of the issues you are seeing with the SDK - and I would welcome any contributions you have towards improvement. The interfaces are a bit clumsy.

The Binding interface is what is causing all of this trouble, I think. https://github.com/cloudevents/sdk-javascript/blob/d9ee0e05d1b38c84314d3e0535b17c25470d95a5/src/message/index.ts#L25-L30

I want this to be generic so that various protocols such as HTTP and Kafka, which have differing implementations can all behave similarly when serializing and deserializing CloudEvent messages. The Serializer and Deserializer interfaces that Binding exposes are meant to be the generic API. https://github.com/cloudevents/sdk-javascript/blob/d9ee0e05d1b38c84314d3e0535b17c25470d95a5/src/message/index.ts#L63-L79

However, the result is what you are seeing now. There is some amount of type casting that is inevitable. If you have ideas or suggestions to improve this situation, I’d be more than happy to talk about them.

0reactions
github-actions[bot]commented, Aug 14, 2022

This issue is stale because it has been open 30 days with no activity.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Structured Streaming Aggregations return wrong values
It uses the Datastax Spark Cassandra connector to write data frames. The problem is that the variable data contains a streaming DataSet.
Read more >
Documentation - Apache Kafka
Each partition is an ordered, immutable sequence of records that is continually appended to—a structured commit log. The records in the partitions are...
Read more >
Apache Kafka Reference Guide - Quarkus
quarkus.reactive-messaging.auto-connector-attachment=false ... The Message type lets the consuming method access the incoming message metadata and handle ...
Read more >
Apache Kafka source in Structured Streaming
By saying data consumer, I mean the consumers from executors responsible for physically polling data from Kafka broker. Can I define only 1 ......
Read more >
Spring for Apache Kafka
poll() method waiting for the topic to appear while logging many messages. Aside from the logs, there was no indication that there was...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found