question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

The kubeconfig loader should run refresh command to update token when it is expired

See original GitHub issue

I am using google container engine, and trying to use this to access the k8s api. Trying to follow the example on the readme

from kubernetes import client, config

config.load_kube_config()
api = client.CoreV1Api()
pods = api.list_pod_for_all_namespaces(watch=False)


for p in pods.items:
    print(p.metadata.name, p.status.phase)

which gives me the following error:

Traceback (most recent call last):
  File "/home/nhumrich/devops/containers/deployment/scripts/kube-deploy.py", line 6, in <module>
    config.load_kube_config()
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 311, in load_kube_config
    client_configuration=client_configuration).load_and_set()
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 216, in load_and_set
    self._load_authentication()
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 155, in _load_authentication
    if self._load_gcp_token():
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 171, in _load_gcp_token
    self.token = "Bearer %s" % self._get_google_credentials()
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/config/kube_config.py", line 124, in <lambda>
    GoogleCredentials.get_application_default()
  File "/home/nhumrich/.local/lib/python3.6/site-packages/oauth2client/client.py", line 1271, in get_application_default
    return GoogleCredentials._get_implicit_credentials()
  File "/home/nhumrich/.local/lib/python3.6/site-packages/oauth2client/client.py", line 1261, in _get_implicit_credentials
    raise ApplicationDefaultCredentialsError(ADC_HELP_MSG)
oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

If I add the GOOGLE_APPLICATION_CREDENTIALS env-var and download a google json credential file, I then get a generic 401.

Traceback (most recent call last):
  File "/home/nhumrich/devops/containers/deployment/scripts/kube-deploy.py", line 19, in <module>
    pods = api.list_pod_for_all_namespaces(watch=False)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/apis/core_v1_api.py", line 13650, in list_pod_for_all_namespaces
    (data) = self.list_pod_for_all_namespaces_with_http_info(**kwargs)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/apis/core_v1_api.py", line 13743, in list_pod_for_all_namespaces_with_http_info
    collection_formats=collection_formats)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 329, in call_api
    _return_http_data_only, collection_formats, _preload_content, _request_timeout)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 153, in __call_api
    _request_timeout=_request_timeout)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 361, in request
    headers=headers)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 240, in GET
    query_params=query_params)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 231, in request
    raise ApiException(http_resp=r)
kubernetes.client.rest.ApiException: (401)
Reason: Unauthorized
HTTP response headers: HTTPHeaderDict({'Content-Type': 'text/plain; charset=utf-8', 'Www-Authenticate': 'Basic realm="kubernetes-master"', 'X-Content-Type-Options': 'nosniff', 'Date': 'Mon, 22 May 2017 21:25:24 GMT', 'Content-Length': '13'})
HTTP response body: Unauthorized

If I try to add an api key (client.configuration.api_key['authorization'] = 'AbX.....SYh' I get another error.

Traceback (most recent call last):
  File "/home/nhumrich/devops/containers/deployment/scripts/kube-deploy.py", line 19, in <module>
    pods = api.list_pod_for_all_namespaces(watch=False)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/apis/core_v1_api.py", line 13650, in list_pod_for_all_namespaces
    (data) = self.list_pod_for_all_namespaces_with_http_info(**kwargs)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/apis/core_v1_api.py", line 13743, in list_pod_for_all_namespaces_with_http_info
    collection_formats=collection_formats)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 329, in call_api
    _return_http_data_only, collection_formats, _preload_content, _request_timeout)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 153, in __call_api
    _request_timeout=_request_timeout)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 361, in request
    headers=headers)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 240, in GET
    query_params=query_params)
  File "/home/nhumrich/.local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 231, in request
    raise ApiException(http_resp=r)
kubernetes.client.rest.ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Content-Type': 'text/plain', 'X-Content-Type-Options': 'nosniff', 'Date': 'Mon, 22 May 2017 21:40:29 GMT', 'Content-Length': '119'})
HTTP response body: User "system:anonymous" cannot list pods at the cluster scope.: "No policy matched.\nUnknown user \"system:anonymous\""

Are there any examples of how I authenticate with kubernetes/google container engine so that I can get this working?

Note: one possible solution is to run gcloud auth application-default login but that isn’t automated and only works locally.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:5
  • Comments:19 (4 by maintainers)

github_iconTop GitHub Comments

10reactions
nhumrichcommented, Jun 12, 2017

Update: I was able to work around this issue by creating a serviceaccount in kubernetes.

I then ran kubectl describe serviceaccount myserviceaccount and that will give you a secret name, then use that secret name to run: kubectl describe secrets [secret-name] and then copy the token field. One you have the token field, all you need to do is set the api token in the client:

config.load_kube_config()
client.configuration.api_key['authorization'] = 'your token goes here'
client.configuration.api_key_prefix['authorization'] = 'Bearer'

This worked great for me. If you dont want to use the kube config file at all, you can also set the host and cert yourself:

client.configuration.api_key['authorization'] = 'your token goes here'
client.configuration.api_key_prefix['authorization'] = 'Bearer'
client.configuration.host = 'https://some.domain-or-ip.example'
client.configuration.ssl_ca_cert = 'cert/location.crt'
2reactions
mooperdcommented, Nov 24, 2018

+1 I’m confused why the library isn’t able to use the Kubeconfig properly.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How can I renew the kube config file expiry date?
Before run this command in my kube config file expiry date was 2022-04-25 and after running above command expiry date get changed to...
Read more >
Kubernetes service accounts - Amazon EKS
Service account tokens have an expiration of one hour. In earlier Kubernetes versions, the tokens didn't have an expiration. This means that clients...
Read more >
Troubleshooting kubeadm | Kubernetes
Execute kubeadm init phase bootstrap-token on a control-plane node using kubeadm v1.18. Note that this enables the rest of the bootstrap-token ...
Read more >
Renewing Kubernetes cluster certificates - IBM
Log on to the Kubernetes master node as the root user and run the following command to check when the Kubernetes certificates will...
Read more >
Redeploying Certificates | OpenShift Container Platform 3.11
The certificates are expired and you need to update them. ... Master, node, router, registry, and kubeconfig files for cluster-admin users.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found