Automatic SQL Instrumentation Unable to Find Spans

Package

Sentry

.NET Flavor

.NET

.NET Version

6.0.0

OS

Windows

SDK Version

6.0.203

Self-Hosted Sentry Version

No response

Steps to Reproduce

I’m facing the issue originally reported in #1210 and was asked to spin up a new ticket. Upon upgrading to the latest version of Sentry, we started seeing a large number of warning log messages from sentry in the form below. Since we had jumped a few versions, I bisected until I found that these warnings started appearing in 3.22.0. I did not see these warning logs in 3.21.0. I can also confirm that disabling automatic SQL instrumentation with DisableDiagnosticSourceIntegration does remove the warning logs.

Trying to get a span of type {type} with operation id {guid}, but it was not found.

Here is an excerpt from our logs that show the sentry warnings. I’m not sure if there is some race condition, because when I set the log level to Trace (Serilog calls this Verbose), I don’t see the sentry warnings, but when the log level is set to Debug, I do.

{"Timestamp":"2022-11-18T10:09:24.3750643-05:00","Level":"Information","MessageTemplate":"Entity Framework Core {version} initialized '{contextType}' using provider '{provider}:{providerVersion}' with options: {options}","Properties":{"version":"6.0.9","contextType":"[REDACTED]","provider":"Microsoft.EntityFrameworkCore.SqlServer","providerVersion":"6.0.9","options":"None","EventId":{"Id":10403,"Name":"Microsoft.EntityFrameworkCore.Infrastructure.ContextInitialized"},"SourceContext":"Microsoft.EntityFrameworkCore.Infrastructure","ActionId":"1e01ba7b-0cc1-4926-82a9-3f9b9b60bec9","ActionName":"[REDACTED]","RequestId":"0HMM9ER7LRNIR:0000000F","RequestPath":"[REDACTED]","ConnectionId":"0HMM9ER7LRNIR","MachineName":"[REDACTED]","ClientIp":"127.0.0.1","ClientAgent":"[REDACTED]"}}
{"Timestamp":"2022-11-18T10:09:24.6612204-05:00","Level":"Warning","MessageTemplate":"Trying to get a span of type {0} with operation id {1}, but it was not found.","Properties":{"0":"Execution","1":"07546a94-0989-4d99-b5f7-c1782184eb80","SourceContext":"Sentry.ISentryClient","ActionId":"1e01ba7b-0cc1-4926-82a9-3f9b9b60bec9","ActionName":"[REDACTED]","RequestId":"0HMM9ER7LRNIR:0000000F","RequestPath":"[REDACTED]","ConnectionId":"0HMM9ER7LRNIR","MachineName":"[REDACTED]","ClientIp":"127.0.0.1","ClientAgent":"[REDACTED]"}}
{"Timestamp":"2022-11-18T10:09:24.6699988-05:00","Level":"Information","MessageTemplate":"Executed DbCommand ({elapsed}ms) [Parameters=[{parameters}], CommandType='{commandType}', CommandTimeout='{commandTimeout}']{newLine}{commandText}","Properties":{"elapsed":"56","parameters":"","commandType":"Text","commandTimeout":30,"newLine":"\r\n","commandText":"SELECT [REDACTED]\r\nFROM [REDACTED]\r\nWHERE [REDACTED]","EventId":{"Id":20101,"Name":"Microsoft.EntityFrameworkCore.Database.Command.CommandExecuted"},"SourceContext":"Microsoft.EntityFrameworkCore.Database.Command","ActionId":"1e01ba7b-0cc1-4926-82a9-3f9b9b60bec9","ActionName":"REDACTED"","RequestId":"0HMM9ER7LRNIR:0000000F","RequestPath":"[REDACTED]","ConnectionId":"0HMM9ER7LRNIR","MachineName":"[REDACTED]","ClientIp":"127.0.0.1","ClientAgent":"[REDACTED]"}}
{"Timestamp":"2022-11-18T10:09:24.7338833-05:00","Level":"Warning","MessageTemplate":"Trying to get a span of type {0} with operation id {1}, but it was not found.","Properties":{"0":"Connection","1":"bcfa705d-1d03-4c85-8ecf-16810e17ea6e","SourceContext":"Sentry.ISentryClient","ActionId":"1e01ba7b-0cc1-4926-82a9-3f9b9b60bec9","ActionName":"REDACTED"","RequestId":"0HMM9ER7LRNIR:0000000F","RequestPath":"[REDACTED]","ConnectionId":"0HMM9ER7LRNIR","MachineName":"[REDACTED]","ClientIp":"127.0.0.1","ClientAgent":"[REDACTED]"}}

Expected Result

I would not expect to see the warning log messages when using entity framework and automatic sql instrumentation enabled with Sentry.

Actual Result

An excessive number of warning log messages.

[vermiceli]

I can confirm that with the 3.25.0 release I am no longer seeing these warning messages. Thank you for you help.

[mattjohnsonpint]

BTW - The .NET SDK guidance about TraceSampleRate needs to be updated. In general, the best results now come from setting 1.0 and relying on Sentry’s Dynamic Sampling feature.