No access to API server from pod in python

Trying to access api service from the pod in python:

from kubernetes import client, config
core_v1 = client.CoreV1Api()
config.load_incluster_config()
core_v1.list_node()

results in following error:

# python
Python 2.7.5 (default, Nov  6 2016, 00:28:07) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> from kubernetes import client, config
>>> core_v1 = client.CoreV1Api()
>>> config.load_incluster_config()
>>> core_v1.list_node()         
2017-06-14 13:52:41,486 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x36c0e50>: Failed to establish a new connection: [Errno 111] Connection refused',)': /api/v1/nodes
2017-06-14 13:52:41,487 WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x36b5b50>: Failed to establish a new connection: [Errno 111] Connection refused',)': /api/v1/nodes
2017-06-14 13:52:41,487 WARNING Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x36b5c90>: Failed to establish a new connection: [Errno 111] Connection refused',)': /api/v1/nodes
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/kubernetes/client/apis/core_v1_api.py", line 13284, in list_node
  File "build/bdist.linux-x86_64/egg/kubernetes/client/apis/core_v1_api.py", line 13377, in list_node_with_http_info
  File "build/bdist.linux-x86_64/egg/kubernetes/client/api_client.py", line 329, in call_api
  File "build/bdist.linux-x86_64/egg/kubernetes/client/api_client.py", line 153, in __call_api
  File "build/bdist.linux-x86_64/egg/kubernetes/client/api_client.py", line 361, in request
  File "build/bdist.linux-x86_64/egg/kubernetes/client/rest.py", line 240, in GET
  File "build/bdist.linux-x86_64/egg/kubernetes/client/rest.py", line 214, in request
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/request.py", line 66, in request
    **urlopen_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/request.py", line 87, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/poolmanager.py", line 321, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/connectionpool.py", line 678, in urlopen
    **response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/connectionpool.py", line 678, in urlopen
    **response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/connectionpool.py", line 678, in urlopen
    **response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/connectionpool.py", line 649, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/lib/python2.7/site-packages/urllib3-1.21.1-py2.7.egg/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='localhost', port=443): Max retries exceeded with url: /api/v1/nodes (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x36b5d90>: Failed to establish a new connection: [Errno 111] Connection refused',))

For some reason HTTPSConnectionPool is trying to access localhost instead of API server, though API url in kubernetes configuration seems to be correct:

>>> from kubernetes.client import configuration
>>> configuration.host
'https://10.0.0.1:443'

At the same time curl is working perfectly well: curl -v --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" https://kubernetes/

[awootton]

I was getting this same error this afternoon from this sequence:

from kubernetes import client, config
config.load_incluster_config()
core_v1 = client.CoreV1Api()
ret = v1.list_namespaced_pod("my-namespace")

(I had some other stuff and several print statements)

No amount of messing about with python libs helped. However, there were the “WARNING Retrying” logs starting right at the beginning. So, importing the client AFTER the config is done worked:

from kubernetes import config
config.load_incluster_config()
from kubernetes import client
core_v1 = client.CoreV1Api()
ret = v1.list_namespaced_pod("my-namespace")

[hayds]

This worked for me:

config.load_incluster_config() #get the config from within the cluster and set it as the default config for all new clients
c=client.Configuration() #go and get a copy of the default config
c.verify_ssl=False #set verify_ssl to false in that config
client.Configuration.set_default(c) #make that config the default for all new clients
v1 = client.CoreV1Api()