question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Could provider exceptions on startup

See original GitHub issue

Describe the bug Hi, when starting my application, I get the three following exceptions thrown by the APM lib

  • AwsCloudMetadataProvider
[10:58:41 INF] {AwsCloudMetadataProvider} Unable to get aws cloud metadata
System.Net.Http.HttpRequestException: Connection refused (169.254.169.254:80)
 ---> System.Net.Sockets.SocketException (111): Connection refused
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|283_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.DefaultConnectAsync(SocketsHttpConnectionContext context, CancellationToken cancellationToken)
   at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
   at Elastic.Apm.Cloud.AwsCloudMetadataProvider.GetMetadataAsync()
  • GcpCloudMetadataProvider
[10:58:41 INF] {GcpCloudMetadataProvider} Unable to get gcp cloud metadata
System.Net.Http.HttpRequestException: Name or service not known (metadata.google.internal:80)
 ---> System.Net.Sockets.SocketException (0xFFFDFFFF): Name or service not known
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|283_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.DefaultConnectAsync(SocketsHttpConnectionContext context, CancellationToken cancellationToken)
   at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
   at Elastic.Apm.Cloud.GcpCloudMetadataProvider.GetMetadataAsync()
  • AzureCloudMetadataProvider
[10:58:42 INF] {AzureCloudMetadataProvider} Unable to get azure cloud metadata
System.Net.Http.HttpRequestException: Connection refused (169.254.169.254:80)
 ---> System.Net.Sockets.SocketException (111): Connection refused
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|283_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.DefaultConnectAsync(SocketsHttpConnectionContext context, CancellationToken cancellationToken)
   at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
   at Elastic.Apm.Cloud.AzureCloudMetadataProvider.GetMetadataAsync()

To Reproduce Steps to reproduce the behavior: My appsettings.json

  "ElasticApm": {
    "ServerUrls": "http://apm:8200",
    "TransactionMaxSpans": 5000,
    "CaptureBody": "all"
  }

My APM extension method

  public static IApplicationBuilder UseElasticSearchAPM(this IApplicationBuilder app, IConfiguration config)
     {
             if (!Elastic.Apm.Agent.IsConfigured)
                 app.UseAllElasticApm(config);
         return app;
     }

My Configure method from Startup.cs

        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            app.UseElasticSearchAPM(Configuration);

Expected behavior I have this problem with only one of my Microservices (.net 5), this one has the same shape as others.

I use the following versions of APM lib

    <PackageReference Include="Elastic.Apm.NetCoreAll" Version="1.7.0" />
    <PackageReference Include="Elastic.Apm.SerilogEnricher" Version="1.5.1" />
    <PackageReference Include="Elastic.CommonSchema.Serilog" Version="1.5.1" />

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
gregkalaposcommented, Nov 18, 2020

@NicolasReyDotNet this is coming from the new cloud detection feature that was introduced in the latest agent release (1.7.0).

The logs are only info logs and they are safe to ignore (all the logged exceptions are handled by the agent and they only happen once). On a default log level you should not see them either.

Here is the documentation of the setting that may help you. The way it works is that the agent tries to discover in which cloud provider the monitored application runs - unfortunately there is no way to know this up-front, so it queries a few known cloud HTTP APIs. If you want to opt out you can set the linked CloudProvider setting to none - with that the agent won’t try to discover in which cloud service the app is running and you won’t get these logs either.

0reactions
wajikacommented, Jan 8, 2022

@NicolasReyDotNet this is coming from the new cloud detection feature that was introduced in the latest agent release (1.7.0).

The logs are only info logs and they are safe to ignore (all the logged exceptions are handled by the agent and they only happen once). On a default log level you should not see them either.

Here is the documentation of the setting that may help you. The way it works is that the agent tries to discover in which cloud provider the monitored application runs - unfortunately there is no way to know this up-front, so it queries a few known cloud HTTP APIs. If you want to opt out you can set the linked CloudProvider setting to none - with that the agent won’t try to discover in which cloud service the app is running and you won’t get these logs either.

It defaults to auto. Why is it recognized as gcp in my environment? I am in China, because of some network problems, I cannot connect to gcp. Every time I start the local debugging code, visual studio reports an error. I didn’t know this configuration at first. I don’t know how “auto” detects the environment, I hope it will be more accurate when set to “auto” or it is recommended to use none by default, which can be turned on when needed.
thanks

Read more comments on GitHub >

github_iconTop Results From Across the Web

Dependency injection in ASP.NET Core 2 throws exception
5 Answers. ASP.NET Core dependency injection provides application services during an application's startup. You can request these services by ...
Read more >
Application not recovering when Startup throws exception
Create a web application and have it throw an exception in the Configure method, but only the very first time the app starts...
Read more >
Handle errors in ASP.NET Core
The Developer Exception Page displays detailed information about unhandled request exceptions. ASP.NET Core apps enable the developer ...
Read more >
Complete Guide to Exception Handling in Spring Boot
This article showcases various ways to handle exceptions in a Spring Boot Application.
Read more >
Unexpected exception during JIRA startup. This JIR...
Hi Community, We have been struggling with our Jira access for a few weeks. Our instance is located in our own server. We...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found