Tracing not working in GKE with Google.Cloud.Diagnostics.AspNetCore 3.0.0-beta13; works locally

Purpose Use Google Cloud Diagnostics on a .net core REST API, for Logging, Tracing and Error Reporting in two possible scenarios:

1. Local execution on Visual Studio 2017
2. Deployed on a Docker Container and Running on a GCP Kubernetes Engine

Description For configuring Google Cloud Diagnostics, two documentation sources were used:

1. Google.Cloud.Diagnostics.AspNetCore
2. Enable configuration support for Google.Cloud.Diagnostics.AspNetCore #2435

Based on the above documentation the UseGoogleDiagnostics extension method on IWebHostBuilder was used, as this configures Logging, Tracing and Error Reporting middleware.

According to the 2) link, the following table presents the information needed when using the UseGoogleDiagnostics method:

1. For local execution => project_id, module_id, and version_id are needed,
2. For GKE => module_id, and version_id

The .net core configuration files were used to provide the above information for each deployment: appsettings.json

{
  "GCP": {
    "ServiceID": "my-service",
    "VersionID": "v1"
  } 
}

appsettings.Development.json

{
  "GCP": {
    "ID": "my-id"
  } 
}

Basically, the above will render the following configuration:

1. On Local execution

return WebHost.CreateDefaultBuilder(args)
	.UseGoogleDiagnostics("my-id", "my-service", "v1")
	.UseStartup<Startup>();

1. On GKE

return WebHost.CreateDefaultBuilder(args)
	.UseGoogleDiagnostics(null, "my-service", "v1")
	.UseStartup<Startup>();

To guarantee i’m using the correct information, i used two places on the GCP UI to verify:

1. On Endpoints Listing, checked the service details:

Service name: my-service Active version: v1 

1. Checked the Endpoint logs, for a specific API POST Endpoint

{
insertId: "e6a63a28-1451-4132-ad44-a4447c33a4ac@a1"  
 
jsonPayload: {…}  
 logName: "projects/xxx%2Fendpoints_log"  
 receiveTimestamp: "2019-07-11T21:03:34.851569606Z"  
 
resource: {
  
labels: {
   location: "us-central1-a"    
   method: "v1.xxx.ApiOCRPost"    
   project_id: "my-id"    
   service: "my-service"    
   version: "v1"    
  }
  type: "api"   
 }
 severity: "INFO"  
 timestamp: "2019-07-11T21:03:27.397632588Z"  
} 

Is there a BUG? When executing the service Endpoints, for each specific deployment, Google Cloud Diagnostics behaves differently:

1. On Local execution (VS2017) => Logging, Tracing and Error Reporting work as expected, everything showing in GCP UI
2. On GKE Deployment => Logging, Tracing and Error Reporting DO NOT Work, nothing shows in GCP UI

I’ve tried several variations, hardcoding the values directly in the code, etc, but no matter what i do, Google Cloud Diagnostics is not working when deployed in GKE:

1. Hardcoding the values directly

return WebHost.CreateDefaultBuilder(args)
	.UseGoogleDiagnostics(null, "my-service", "v1")
	.UseStartup<Startup>();

1. Without the v in Version

return WebHost.CreateDefaultBuilder(args)
	.UseGoogleDiagnostics(null, "my-service", "1")
	.UseStartup<Startup>();

Am i doing anything wrong or is it a bug on Google.Cloud.Diagnostics.AspNetCore 3.0.0-beta13?

Environment details

• .NET version: 2.2.0
• Package name and version: Google.Cloud.Diagnostics.AspNetCore 3.0.0-beta13

[jskeet]

Hooray! I’m really glad we got there in the end. Thanks for all your patience 😃

[jskeet]

I’ve finished work for the day, but I’ll look tomorrow. I’m aware of some issues with resource detection in GKE (as linked earlier) but I don’t know whether that’s what’s going wrong. Will look into it more. I’m glad it’s working when everything is specified though.