question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Pub Sub: How to get meta-data about the subscription

See original GitHub issue

For a project I’m working on, I would like to use Dapr to subscribe for messages that are being sent to an MQTT broker. I’ve been looking at the documentation of how this can be done in .NET and it looks like the way to go is to create an ASP.NET application and create a controller method that acts as the subscriber, just like this:

[Topic("mypubsub", "vesselmessages")]
[HttpPost("/vesselmessage")]
public async Task<ActionResult> ReceiveMessages([FromBody] VesselMessage message)
{
      _logger.LogInformation("message received: " + message.Message);
         
       return Ok();
}

This works, but there are some things that I’d like to achieve, but I cannot find anything about that in the documentation. (Maybe I overlooked something):

  • In fact, I want to subscribe to a wildcards of topics. so I’ve changed my topic subscription to:

    [Topic("mypubsub", "vesselmessages/#")]
    

    And this works. I can see that if I send a message to topic vesselmessages/test, my asp.net application receives the message.

    But now, when I receive a message, I want to know on which exact topic that message was received. So I’d like to be able to receive some meta-data about the message next to the message as well. How can I achieve that ?

  • It looks like the way the .NET sdk works for pub/sub, is that it expects JSON formatted messages. However, I’d like to receive the raw payload in my subscription-handler in ASP.NET. I need this since the messages that I want to receive, are not always JSON formatted messages. I have devices that sent messages to the MQTT broker in a protobuf format. Therefore, I want to receive the raw payload so that I can deserialize them myself. Is that possible ?

Issue Analytics

  • State:open
  • Created 7 months ago
  • Comments:6 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
modabascommented, Mar 26, 2023

As far as I know, dapr pub sub consumer assumes received message is in cloudevent format by default, as a message published by a dapr client is wrapped in a cloudevent unless specified otherwise.

Can you try to add rawPayload metadata while registering ListTopicSubscriptions, so received events will not be processed as cloudevents.

public override Task<ListTopicSubscriptionsResponse> ListTopicSubscriptions(Empty request, ServerCallContext context)
{
            var result = new ListTopicSubscriptionsResponse();
            var ts = new TopicSubscription()
            {
                PubsubName = "telemetrypubsub",
                Topic = "vesseltest",
            };
            ts.Metadata.Add("rawPayload", "true");
            result.Subscriptions.Add(ts);

//rest of the code...

0reactions
modabascommented, Apr 1, 2023

@fgheysels, glad it worked out for you.

@halspang, as fgheysels suggested documentation on this would be nice. I have put together a sample .net grpc service to consume messages published by non-dapr services. You can add it to dapr datnet-sdk examples if you like.

It utilizes RabbitMq as pubsub component, and even though it’s possible to get and process raw message, I couldn’t find a way to read message headers from either TopicEventRequest or ServerCallContext parameters. Is there a way to get message headers?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Subscription properties | Cloud Pub/Sub Documentation
This option lets Pub/Sub write the metadata of each message to additional columns in the BigQuery table. Else, the metadata is not written...
Read more >
How to get the metadata of Pub/Sub event in Spring Cloud ...
I have created a simple Google Cloud Function with Spring Cloud Function library to get triggered on the arrival of the Pub/Sub message....
Read more >
Solved: Pub / Sub Filter by Custom Metadata
Solved: Hi All, Wondering if it is possible to filter pub / sub messages by custom metadata fields on files for cloud storage...
Read more >
Pass pubsub subscription metadata to pubsub component
Adding metadata from the subscription is easy enough to the SubscribeRequest . However, the data in PublishRequest comes directly from the user ...
Read more >
Pub/sub API reference
Note, all subscription parameters are case-sensitive. Metadata. Optionally, metadata can be sent via the request body. Parameter, Description ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found