Missing Dependencies when Using Microsoft.Azure.ServiceBus

APM Agent version

<PackageReference Include="Azure.Storage.Blobs" Version="12.8.4" />
<PackageReference Include="Elastic.Apm" Version="1.11.0" />
<PackageReference Include="Elastic.Apm.Azure.Storage" Version="1.11.0" />

Environment

.NET 5.0

Describe the bug

I have a service that uses NServiceBus, which internally uses Microsoft.Azure.ServiceBus. I use UseAllElasticApm however the only thing I am seeing in APM is AzureServiceBus RECEIVE from xxxx in the transactions section and nothing in dependencies.

I think this may be a similar issue to https://github.com/elastic/apm-agent-dotnet/issues/1321, as I have managed to reproduce the issue where CurrentTransaction is null. Manually creating the transaction solves the issue but is not ideal.

Here is the code I used:

using System;
using System.Threading;
using System.Threading.Tasks;
using Elastic.Apm.Api;
using Elastic.Apm.NetCoreAll;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Azure.ServiceBus.Core;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

namespace ApmServiceBusTest
{
    internal class MessageHandler : IHostedService
    {
        private readonly IServiceProvider _serviceProvider;
        private readonly string cnnString = "";
        private readonly string queueName = "";
        
        public MessageHandler(IServiceProvider serviceProvider)
        {
            _serviceProvider = serviceProvider;
        }

        public async Task StartAsync(CancellationToken cancellationToken)
        {
            var receiver = new MessageReceiver(cnnString, queueName);
            using var scope = _serviceProvider.CreateScope();
            var tracer = _serviceProvider.GetService<ITracer>();
            
            while (true)
            {
                await receiver.ReceiveAsync();
                var transaction = tracer.CurrentTransaction;
            }
        }

        public Task StopAsync(CancellationToken cancellationToken)
        {
            return Task.CompletedTask;
        }
    }

    internal class Startup
    {
        public static void ConfigureServices(IServiceCollection services)
        {
            services.AddHostedService<MessageHandler>();
        }
        
        public static void Configure() 
        {
            
        }
    }

    internal static class Program
    {
        private static IHostBuilder CreateHostBuilder(string[] args)
        {
            return Host.CreateDefaultBuilder(args)
                .ConfigureWebHostDefaults(webBuilder => { webBuilder.UseStartup<Startup>(); })
                .UseAllElasticApm();
        }

        public static void Main(string[] args)
        {
            CreateHostBuilder(args).Build().Run();
        }
    }
}
`

[waczi]

@gregkalapos @russcam

I checked it again and it looks like you are right. I can see PROCESS transaction which contains child spans as expected. When I was testing it previously I hade to miss it because I assumed that there is no separation into two transactions while handling service bus message. Sorry sorry for the misunderstanding.

Regards.

[russcam]

Thanks for opening @gardnerr, and thanks for your input, @waczi. Let me see if I can clear up how the Azure Service Bus integration works.

What you’re describing @gardnerr sounds like the expected behaviour and the same as https://github.com/elastic/apm-agent-dotnet/issues/1321. It’d be good to discuss whether this is serving you and other users well though. Given this has come up a couple of times, I’d like to see if we can improve things.

The Azure Service Bus integration follows the APM messaging spec. The intent of message reception is to capture a transaction around the message handling flow. When using a processor or session processor like ServiceBusProcessor in Azure.Messaging.ServiceBus, or registering a message handler in Microsoft.Azure.ServiceBus, the message handling flow is captured in a transaction with a name starting with AzureServiceBus PROCESS from <queue/topic subscription>. I put together a sample app based on @waczi’s example (happy to share if it helps), using Microsoft.Azure.ServiceBus 3.0.0, and see that the message handling transactions are captured, as is the span for the HTTP call that happens as part of the transaction:

Now, if we look at the messaging type transactions, we see there are two differently named transactions captured

The AzureServiceBus RECEIVE from <queue/topic subscription> transactions relate to receiving message(s) from Azure Service Bus; a transaction is started when the Receive Start activity is seen, and stopped when the Receive Stop activity is seen. As is, this is probably not very useful, since these activities are raised one after another, and from a user’s code perspective, correlate with the ReceiveAsync() call, i.e.

await receiver.ReceiveAsync(); // Starts and ends a transaction

If not using a processor type or registering a message handler, the ideal message handling flow would be:

{
    await receiver.ReceiveAsync(); // Start a transaction

    // do some work that results in captured spans

    // end the transaction
}

Unfortunately, it’s not possible to do this automatically with the activities raised, when receiving messages manually from Azure Service Bus. It may make sense to capture the ReceiveAsync() calls as a span if there is a current transaction, to allow the call to be traced when manually calling ReceiveAsync(). This would require a transaction to be manually started, which is not ideal, but perhaps better than the current implementation?

When using the processors or message handlers, it is possible to capture the messaging flow, since the activities raised by the libraries follow the pattern

Receive.Start                // start a transaction
Receive.Stop                 // end transaction
ProcessMessage.Start         // start a transaction

/** inside HandleMessage processor delegate **/

Complete.Start
Complete.Stop
ProcessMessage.Stop         // end transaction

If we were to capture the ReceiveAsync() calls as a span however, we might not be able to capture it when it’s part of a processor or message handler, since we would need an active transaction.