question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Resource: Sanitize the "location" label for k8s resource.

See original GitHub issue

For the “location” label of k8s resource, we used to get the value from GcpMetadataConfig.getZone() and pass it directly to Stackdriver Monitoring, and it worked fine previously. However it looks like the format of GCP metadata Zone has changed recently: (https://github.com/googleapis/gax-dotnet/issues/233):

  • Before: “us-central1-d”
  • Now: “projects/…/zones/us-central1-d”

If we directly passed the new value to Stackdriver, an error will be returned saying that com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: One or more TimeSeries could not be written: Unrecognized region or location.: ....

This can be fixed by only extracting the location value from the new label.

Issue Analytics

  • State:open
  • Created 5 years ago
  • Comments:6 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
songy23commented, Jan 29, 2019

@samhaink Could you try ssh to one of the container and see if you’re able to retrieve the location info? Use wget --header "Metadata-Flavor: Google" "http://metadata.google.internal/computeMetadata/v1/instance/instance/zone".

@mayurkale22 has tried to reproduce this error but it looks like everything is working fine with our GKE instance.

0reactions
songy23commented, Jan 31, 2019

@samhaink Apologies, the second url is correct. Looks like you’re able to get the zone string (projects/*/zones/us-east1-c). With our resource util, “us-east1-c” should be auto-detected and added as “location” in the resource labels (https://github.com/census-instrumentation/opencensus-java/blob/master/contrib/monitored_resource_util/src/main/java/io/opencensus/contrib/monitoredresource/util/GcpMetadataConfig.java#L46-L55).

This looks strange. We’ll try to reproduce this error again.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Labels and Selectors - Kubernetes
Labels are key/value pairs that are attached to objects, such as pods. Labels are intended to be used to specify identifying attributes of ......
Read more >
Sanitizing Kubernetes resources into strings #332 - GitHub
Some Kubernetes resources such as V1EnvVar for container environment variables only accept string, so we need to let the SDK to sanitize ......
Read more >
Use the KubernetesPodOperator | Cloud Composer
This page describes how to use the KubernetesPodOperator to launch Kubernetes pods from Cloud Composer into the Google Kubernetes Engine cluster that is...
Read more >
Manage Kubernetes Resources via Terraform
You can use the Terraform Kubernetes provider to interact with resources ... resource block dynamically assigns the selector to the Deployment's label.
Read more >
Running Spark on Kubernetes - Spark 3.3.1 Documentation
You can verify that you can list these resources by running kubectl auth can-i ... This URI is the location of the example...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found