Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

kubernetes.client.rest.ApiException: (503) error with list_cluster_custom_object

See original GitHub issue

What happened (please include outputs or screenshots): I wanted to use client to get node data collected by metric-server and set it to be collected every 30 seconds, but the following error occurred after several times of collection

Error is as follows

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/test.py", line 51, in get_host_info
    for device_resource_info in api.list_cluster_custom_object("metrics.k8s.io", "v1beta1", "nodes")['items']:
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api/custom_objects_api.py", line 1344, in list_cluster_custom_object
    (data) = self.list_cluster_custom_object_with_http_info(group, version, plural, **kwargs)  # noqa: E501
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api/custom_objects_api.py", line 1457, in list_cluster_custom_object_with_http_info
    collection_formats=collection_formats)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 345, in call_api
    _preload_content, _request_timeout)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 176, in __call_api
    _request_timeout=_request_timeout)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/api_client.py", line 366, in request
    headers=headers)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 241, in GET
    query_params=query_params)
  File "/usr/local/lib/python3.6/site-packages/kubernetes/client/rest.py", line 231, in request
    raise ApiException(http_resp=r)
kubernetes.client.rest.ApiException: (503)
Reason: Service Unavailable
HTTP response headers: HTTPHeaderDict({'Content-Type': 'text/plain; charset=utf-8', 'X-Content-Type-Options': 'nosniff', 'Date': 'Wed, 27 May 2020 14:03:55 GMT', 'Content-Length': '20'})
HTTP response body: service unavailable

What you expected to happen: I want to be able to get node’s resource status every time.

How to reproduce it (as minimally and precisely as possible): The client part of the code is as follows

config.kube_config.load_kube_config(config_file="~/.kube/config")
v1 = client.CoreV1Api()
api = client.CustomObjectsApi()
while(true):
    for node_resource_info in api.list_cluster_custom_object("metrics.k8s.io", "v1beta1", "nodes")['items']:
    ...
    ...
    time.sleep(30)

Anything else we need to know?:

Environment:

Kubernetes version (kubectl version): v1.17.3
OS (e.g., MacOS 10.13.6): The container OS is Alpine Linux 3.10 and the host OS is Ubuntu 16.04.6 LTS
Python version (python --version): python3.6.9
Python client version (pip list | grep kubernetes)：kubernetes==11.0.0

Issue Analytics

State:
Created 3 years ago
Comments:12 (8 by maintainers)

Top GitHub Comments

1reaction

roycaihwcommented, Jun 9, 2020

Yes this looks more like a server-side issue than a python client one. You could enable the debug logging to see if the server gives any more meaningful response, or try monitoring the same endpoint with kubectl kubectl get --raw '/apis/metrics.k8s.io/v1beta1/nodes'

0reactions

k8s-ci-robotcommented, Sep 8, 2020

@MoShitrit: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.