Get secret from keyvault hangs when using with a proxy

• Package Name: azure-keyvault-secrets, azure-identidy, azure-core
• Package Version: azure-keyvault-secrets==4.3.0, azure-identidy==1.6.1, azure-core==2.12.1, requests==2.27.1, urllib3==1.26.9
• Operating System: Ubuntu 18.04
• Python Version: 3.9.12

Describe the bug My bug is similar to #23623 My teams encouters a problem when using azure python sdk with a proxy. The python method to get the secret is stuck forever. The strange thing is that with azure CLI, it works, but with the python SDK it does’nt

To Reproduce First setup env for the proxy and connect to azure.

export HTTP_PROXY=http://<ip>:<port>
export HTTPS_PROXY=http://<ip>:<port>
az login

Then launch this python script

from azure.keyvault.secrets import SecretClient
from azure.identity import DefaultAzureCredential

cred = DefaultAzureCredential()
secret_client = SecretClient("https://<my_vault>.vault.azure.net/", credential=cred)
secret_client.get_secret("test")

Here the script is stucked and whe must CTRL+C to quit this state Output Traceback:

EnvironmentCredential.get_token failed: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
^CTraceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/tracing/decorator.py", line 83, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/keyvault/secrets/_client.py", line 67, in get_secret
    bundle = self._client.get_secret(
  File "/home/ems/.local/lib/python3.9/site-packages/azure/keyvault/secrets/_generated/_operations_mixin.py", line 1515, in get_secret
    return mixin_instance.get_secret(vault_base_url, secret_name, secret_version, **kwargs)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/keyvault/secrets/_generated/v7_2/operations/_key_vault_client_operations.py", line 286, in get_secret
    pipeline_response = self._client._pipeline.run(request, stream=False, **kwargs)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 211, in run
    return first_node.send(pipeline_request)  # type: ignore
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  [Previous line repeated 2 more times]
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/policies/_redirect.py", line 158, in send
    response = self.next.send(request)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/policies/_retry.py", line 445, in send
    response = self.next.send(request)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/keyvault/secrets/_shared/challenge_auth_policy.py", line 111, in send
    self._handle_challenge(request, challenge)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/keyvault/secrets/_shared/challenge_auth_policy.py", line 137, in _handle_challenge
    self._token = self._credential.get_token(scope)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/identity/_credentials/default.py", line 150, in get_token
    return super(DefaultAzureCredential, self).get_token(*scopes, **kwargs)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/identity/_credentials/chained.py", line 67, in get_token
    token = credential.get_token(*scopes, **kwargs)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/identity/_internal/decorators.py", line 27, in wrapper
    token = fn(*args, **kwargs)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/identity/_credentials/managed_identity.py", line 88, in get_token
    return self._credential.get_token(*scopes, **kwargs)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/identity/_internal/get_token_mixin.py", line 72, in get_token
    token = self._request_token(*scopes)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/identity/_credentials/imds.py", line 79, in _request_token
    token = self._client.request_token(*scopes, headers={"Metadata": "true"})
  File "/home/ems/.local/lib/python3.9/site-packages/azure/identity/_internal/managed_identity_client.py", line 123, in request_token
    response = self._pipeline.run(request, retry_on_methods=[request.method], **kwargs)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 211, in run
    return first_node.send(pipeline_request)  # type: ignore
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/policies/_retry.py", line 445, in send
    response = self.next.send(request)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 71, in send
    response = self.next.send(request)
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/_base.py", line 103, in send
    self._sender.send(request.http_request, **request.context.options),
  File "/home/ems/.local/lib/python3.9/site-packages/azure/core/pipeline/transport/_requests_basic.py", line 273, in send
    response = self.session.request(  # type: ignore
  File "/home/ems/.local/lib/python3.9/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ems/.local/lib/python3.9/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/home/ems/.local/lib/python3.9/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/home/ems/.local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/home/ems/.local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/ems/.local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.9/http/client.py", line 1377, in getresponse
    response.begin()
  File "/usr/lib/python3.9/http/client.py", line 320, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.9/http/client.py", line 281, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
KeyboardInterrupt

Additional context As I says, it worked with azure CLI, the folowing works perfectly fine

az login
az keyvault secret list --id https://<my_vault>.vault.azure.net/

Base on other issue #23623, Our teams already tried the following:

import requests
requests.get('http://bing.com')
#It works

from azure.identity import DefaultAzureCredential
cred = DefaultAzureCredential()
cred.get_token("https://vault.azure.net/.default")
#It does'nt work and is stucked too

I will just add that the proxy was set up by our IT organisation and not by us

[QHivert]

Hello, thanks for your quick replies.

You were effective because you brought us two solutions ^^

From @mccoyp

from azure.identity import AzureCliCredential
cred = AzureCliCredential()
cred.get_token("https://vault.azure.net/.default")

I’ve tried this solution and it worked ! I explore further by doing this:

from azure.identity import DefaultAzureCredential

exclude_credentials = {
"exclude_environment_credential": True,
"exclude_shared_token_cache_credential": True,
"exclude_visual_studio_code_credential": True,
"exclude_interactive_browser_credential": True,
"exclude_powershell_credential": True,
"exclude_managed_identity_credential": True
}
cred = DefaultAzureCredential(**exclude_credentials ) #Only keep CLI Credential
cred.get_token("https://vault.azure.net/.default") #Hopefully It works!!

exclude_credentials = {
"exclude_managed_identity_credential": True
}
cred = DefaultAzureCredential(**exclude_credentials ) #Only pop Managed Identity Credential
cred.get_token("https://vault.azure.net/.default") #Still works!!

So the hanging part seems to come from Managed Identity Credential. Which is too bad because our application can use CLI or Managed Identity depending on its environnement.

Then, as for @xiangyan99 answer: Without editing no_proxy, it hanged as expected

ems@ems-fake-quentin21:~$ curl 'http://169.254.169.254/metadata/identity/oauth2/token' -v
* Uses proxy env variable no_proxy == 'localhost,127.0.0.0/8,::1'
* Uses proxy env variable http_proxy == 'http://10.81.35.250:8080/'
*   Trying 10.81.35.250:8080...
* TCP_NODELAY set
* Connected to 10.81.35.250 (10.81.35.250) port 8080 (#0)
> GET http://169.254.169.254/metadata/identity/oauth2/token HTTP/1.1
> Host: 169.254.169.254
> User-Agent: curl/7.68.0
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
^C

Adter adding 169.254.169.254 to no_prox, the curl didn’t hang

ems@ems-fake-quentin21:~$  curl 'http://169.254.169.254/metadata/identity/oauth2/token' -v
* Uses proxy env variable no_proxy == 'localhost,169.254.169.254,127.0.0.0/8,::1'
*   Trying 169.254.169.254:80...
* TCP_NODELAY set
* connect to 169.254.169.254 port 80 failed: No route to host
* Failed to connect to 169.254.169.254 port 80: No route to host
* Closing connection 0
curl: (7) Failed to connect to 169.254.169.254 port 80: No route to host

Then using DefaultAzureCredential worked without hanging!

Our team will go with the no_proxy solution as it allows to not change our code depenfing of the environnement. Thanks a lot ! I’ll close this issue but if you want me to do more test to more uderstand what happens don’t hesitate to ask.

[xiangyan99]

It looks like your app stuck when connecting to imds.

Can you try

curl ‘http://169.254.169.254/metadata/identity/oauth2/token’ -v

to validate it? (It is expected to hang too).

If this is the case, a temporary solution is: Add 169.254.169.254 to your no_proxy environment variable.