Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

az rest command opening json body as latin-1 instead of utf-8

See original GitHub issue

Related command

powershell>az rest --body `@myfile.json

Describe the bug az cli opening the json file without using utf-8 encoding

ERROR: The command failed with an unexpected error. Here is the traceback:                                           ]
ERROR: 'latin-1' codec can't encode character '\u2584' in position 25229: Body ('▄') is not valid Latin-1. Use body.encode('utf-8') if you want to send it encoded in UTF-8.
Traceback (most recent call last):
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 231, in invoke
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 658, in execute
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 721, in _run_jobs_serially
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 692, in _run_job
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 328, in __call__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/util/custom.py", line 23, in rest_call
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/util.py", line 973, in send_raw_request
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\requests/sessions.py", line 655, in send
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\requests/adapters.py", line 439, in send
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\urllib3/connectionpool.py", line 699, in urlopen
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\urllib3/connectionpool.py", line 394, in _make_request
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\urllib3/connection.py", line 239, in request
  File "http\client.py", line 1282, in request
  File "http\client.py", line 1327, in _send_request
  File "http\client.py", line 166, in _encode
UnicodeEncodeError: 'latin-1' codec can't encode character '\u2584' in position 25229: Body ('▄') is not valid Latin-1. Use body.encode('utf-8') if you want to send it encoded in UTF-8.

To Reproduce

Have a json with W▄RTTEMBERG as a value somewhere

Expected behavior No error

Environment summary

azure-cli                         2.36.0 *

core                              2.36.0 *
telemetry                          1.0.6

Extensions:
azure-devops                      0.25.0
costmanagement                     0.2.1
databricks                         0.8.0
resource-graph                     2.1.0

Dependencies:
msal                              1.17.0
azure-mgmt-resource               20.0.0

Python location 'C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe'
Extensions directory 'C:\Users\me\.azure\cliextensions'

Python (Windows) 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 22:57:10) [MSC v.1929 32 bit (Intel)]

Legal docs and information: aka.ms/AzureCliLegal

Additional context

Trying to avoid https://github.com/Azure/azure-cli/issues/10339 by using a json file instead of inlining the json in the command

https://stackoverflow.com/questions/29354602/powershell-error-the-command-line-is-too-long

https://stackoverflow.com/questions/63413704/azure-cli-rest-issue-with-encoding-german-umlauts

Issue Analytics

State:
Created a year ago
Comments:5 (1 by maintainers)

Top GitHub Comments

1reaction

navba-MSFTcommented, Jun 6, 2022

@TeamDman Apologies for the late reply. I think it’s a platform issue related to the unique combination Windows + Python.

Could you please try to update the Windows system language setting ?

Control panel -> Clock and Region -> Region -> Administrative -> Change system locale -> Check Beta: Use Unicode UTF-8 for worldwide language support.

The change will require reboot. Please let me know if you still face the same issue. Awaiting your reply.

0reactions

ZeChArtiahSahercommented, Dec 2, 2022

Doesn’t work on posix az-cli. Just add body encoding arg cmon

Top Results From Across the Web

Azure CLI Rest issue with encoding (German Umlauts)

The server fails to decode the ISO-8859-1 byte FC as ü , and replaces it with the "REPLACEMENT CHARACTER" instead, and finally encodes...

Tips for using the Azure CLI successfully - Microsoft Learn

REST API commands (az rest) The command automatically authenticates using the logged-in credential and sets header Content-Type: application/ ...

MySQL encoding hell: How to export UTF-8 data from a latin1 ...

This post talks about the real problem going underneath the cushy MySQL cover, and more important tells you how to solved it.

exiftool Application Documentation

A command-line interface to Image::ExifTool, used for reading and writing ... When writing, -L specifies that input text values are Latin1 instead of...

https://perldoc.perl.org/5.12.5/perlunicode.txt

To indicate that Perl source itself is in UTF-8, use C<use utf8;>. ... in I<ISO 8859-1 (Latin-1)>, but Unicode strings are downgraded with...