SSL: CERTIFICATE_VERIFY_FAILED error on "az bicep install"

Describe the bug

Some bicep commands generate a SSL: CERTIFICATE_VERIFY_FAILED error.

Command Name az bicep install

Error: Error while attempting to retrieve the latest Bicep version: HTTPSConnectionPool(host=‘api.github.com’, port=443): Max retries exceeded with url: /repos/Azure/bicep/releases/latest (Caused by SSLError(SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)’))).

With the --debug parameter cli.knack.cli: Command arguments: [‘bicep’, ‘install’, ‘–debug’] cli.knack.cli: init debug log: Enable color in terminal. Init colorama. cli.knack.cli: Event: Cli.PreExecute [] cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x02DD41D8>, <function OutputProducer.on_global_arguments at 0x02F51070>, <function CLIQuery.on_global_arguments at 0x02F67C40>] cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate [] cli.azure.cli.core: Modules found from index for ‘bicep’: [‘azure.cli.command_modules.resource’] cli.azure.cli.core: Loading command modules: cli.azure.cli.core: Name Load Time Groups Commands cli.azure.cli.core: resource 0.045 39 182 cli.azure.cli.core: Total (1) 0.045 39 182 cli.azure.cli.core: Loaded 39 groups, 182 commands. cli.azure.cli.core: Found a match in the command table. cli.azure.cli.core: Raw command : bicep install cli.azure.cli.core: Command table: bicep install cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x033030B8>] cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to ‘C:\Users[redacted].azure\commands\2021-09-01.14-26-27.bicep_install.19064.log’. az_command_data_logger: command args: bicep install --debug cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x03347220>, <function register_global_query_examples_argument.<locals>.register_query_examples at 0x033578E0>] cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad [] cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x03357928>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x033579B8>] cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded [] cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [] cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x02F510B8>, <function CLIQuery.handle_query_parameter at 0x02F67C88>, <function register_global_query_examples_argument.<locals>.handle_example_parameter at 0x033472B0>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x03357970>] urllib3.connectionpool: Starting new HTTPS connection (1): api.github.com:443 cli.azure.cli.core.util: azure.cli.core.util.handle_exception is called with an exception: cli.azure.cli.core.util: Traceback (most recent call last): File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\urllib3/connectionpool.py”, line 696, in urlopen File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\urllib3/connectionpool.py”, line 964, in prepare_proxy File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\urllib3/connection.py”, line 411, in connect File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\urllib3/util/ssl.py", line 449, in ssl_wrap_socket File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\urllib3/util/ssl_.py”, line 493, in _ssl_wrap_socket_impl File “ssl.py”, line 500, in wrap_socket File “ssl.py”, line 1040, in _create File “ssl.py”, line 1309, in do_handshake ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\requests/adapters.py”, line 439, in send File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\urllib3/connectionpool.py”, line 755, in urlopen File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\urllib3/util/retry.py”, line 574, in increment urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host=‘api.github.com’, port=443): Max retries exceeded with url: /repos/Azure/bicep/releases/latest (Caused by SSLError(SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)’)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/resource/_bicep.py”, line 151, in get_bicep_latest_release_tag File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\requests/api.py”, line 76, in get File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\requests/api.py”, line 61, in request File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\requests/sessions.py”, line 542, in request File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\requests/sessions.py”, line 655, in send File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\requests/adapters.py”, line 514, in send requests.exceptions.SSLError: HTTPSConnectionPool(host=‘api.github.com’, port=443): Max retries exceeded with url: /repos/Azure/bicep/releases/latest (Caused by SSLError(SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)’)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py”, line 231, in invoke File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py”, line 657, in execute File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py”, line 720, in _run_jobs_serially File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py”, line 691, in _run_job File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py”, line 328, in call File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py”, line 121, in handler File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/resource/custom.py”, line 3522, in install_bicep_cli File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/resource/_bicep.py”, line 103, in ensure_bicep_installation File “D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/resource/_bicep.py”, line 155, in get_bicep_latest_release_tag azure.cli.core.azclierror.ClientRequestError: Error while attempting to retrieve the latest Bicep version: HTTPSConnectionPool(host=‘api.github.com’, port=443): Max retries exceeded with url: /repos/Azure/bicep/releases/latest (Caused by SSLError(SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)’))).

cli.azure.cli.core.azclierror: Error while attempting to retrieve the latest Bicep version: HTTPSConnectionPool(host=‘api.github.com’, port=443): Max retries exceeded with url: /repos/Azure/bicep/releases/latest (Caused by SSLError(SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)’))). az_command_data_logger: Error while attempting to retrieve the latest Bicep version: HTTPSConnectionPool(host=‘api.github.com’, port=443): Max retries exceeded with url: /repos/Azure/bicep/releases/latest (Caused by SSLError(SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)’))). cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x033031D8>] az_command_data_logger: exit code: 1 cli.main: Command ran in 0.981 seconds (init: 0.397, invoke: 0.585) telemetry.save: Save telemetry record of length 3364 in cache

To Reproduce

In a PowerShell Core window, type: az bicep install

Expected behavior

The bicep component should install.

Environment summary

Corporate proxy

The development machine is behind a corporate proxy. The certificates for this proxy are in the Windows certificate store and also pasted in C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem

The instructions in https://docs.microsoft.com/en-us/cli/azure/use-cli-effectively#work-behind-a-proxy have been applied (without these, Azure CLI wouldn’t work in the first place).

Az CLI

az --version azure-cli 2.27.2

core 2.27.2 telemetry 1.0.6

Python location ‘C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe’ Extensions directory ‘C:\Users[redacted].azure\cliextensions’

Python (Windows) 3.8.9 (tags/v3.8.9:a743f81, Apr 6 2021, 13:22:56) [MSC v.1928 32 bit (Intel)]

The REQUESTS_CA_BUNDLE environment variable has been set.

PowerShell

PowerShell 7.1.4

Windows

Microsoft Windows [Version 10.0.14393]

Additional context

Important to know is that other commands that requires the proxy certificate do work, for example: az login works as expected. If the certificate wasn’t available, this would fail as well.

A workaround is to download the bicep cli manually form https://github.com/Azure/bicep/releases/tag/v0.4.613 and put the executable in the .Azure/bin folder.

It looks like it is similar to issue https://github.com/Azure/bicep/issues/3147 that was closed. The solution in the comments from august 28th didn’t work on my machine.

To me, it looks like a small part of the Azure CLI code does not look in the provided certificate file (C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem). Again, many other commands that go through the proxy do work.

Maybe related to: https://github.com/Azure/azure-cli/issues/15121 https://github.com/Azure/azure-cli/issues/14858

It is not just on one machine, every colleague in my corporation I’ve spoken has the same issue.

[jtrivett]

Seeing this issue on 2.33.0 - normal az commands work perfectly fine so long as REQUESTS_CA_BUNDLE environment variable is set. Does bicep not respect this like the CLI?

Yes, it seems to be a bug/oversight in the az cli. See more details and workarounds posted in issue #20842

[baileydoestech]

Seeing this issue on 2.33.0 - normal az commands work perfectly fine so long as REQUESTS_CA_BUNDLE environment variable is set. Does bicep not respect this like the CLI?