Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

UnicodeDecodeError during parsing Config file

See original GitHub issue

We had an repository using GITCRYPT for the config file. As a result, during the parsing of repo in Studio, we had the error as below:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x93 in position 17: invalid start byte
......
    config = dict(Config(dvc_dir, validate=False))
  File "dvc/config.py", line 105, in __init__
    self.load(validate=validate, config=config)
  File "dvc/config.py", line 151, in load
    conf = self.load_config_to_level()
  File "dvc/config.py", line 283, in load_config_to_level
    merge(merged_conf, self.load_one(merge_level))
  File "dvc/config.py", line 200, in load_one
    conf = self._load_config(level)
  File "dvc/config.py", line 179, in _load_config
    conf_obj = ConfigObj(fobj)
  File "configobj.py", line 1229, in __init__
    self._load(infile, configspec)
  File "configobj.py", line 1279, in _load
    content = infile.read() or []
  File "codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)

I think it should be wrapped by ConfigError.

More context at https://sentry.io/organizations/iterative/issues/3606551247/?project=5220519&referrer=slack#exception

DVC Version

2.24.0

Issue Analytics

State:
Created a year ago
Reactions:1
Comments:7 (7 by maintainers)

Top GitHub Comments

1reaction

Suorcommented, Sep 29, 2022

@daavoo wrapping with a ConfigError will help. We automatically show such messages to a user, for now it’s an “unexpected error”.

1reaction

Suorcommented, Sep 27, 2022

I don’t think it’s possible it will work with a gitcrypted config or anything else for that matter. If we don’t have an encryption key there is no way for us to read it.

The possible cases for “worked before” are:

It worked before gitcrypt was used on the same repo.
gitcrypt was used but the history was not encrypted only newer commits, which might not event exist, or be only in some branches. Then someone encrypted and rewrote the history.
gitcrypt was used, but the config file itself was added before and was not altered since. Someone updated the config file and it was encrypted.

I never used gitcrypt so no big expert here, but at least option 1 is possible.

Top Results From Across the Web

While reading file on Python, I got a UnicodeDecodeError ...

Try: enc = 'utf-8' log = open('/Users/Owner/Desktop/Exodus Logs/DIRTYLOGS/serverlog_20130430_00.15.21.txt', 'r', encoding=enc).

Detect UnicodeDecodeErrors when parsing config file #609

If a config file has characters that we cannot handle, we should catch the UnicodeDecodeError and raise a new error to help the...

Processing Text Files in Python 3 - Nick Coghlan's Python Notes

UnicodeDecodeError may be thrown when reading such files (if the data is not actually in the specified encoding); UnicodeEncodeError may be thrown when...

ConfigParser – Work with configuration files - PyMOTW

Configuration files containing Unicode data should be opened using the codecs module ... import codecs parser = SafeConfigParser() # Open the file with...

Miniconda unicode decode error in latest fmriprep singularity ...

File “/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/parser.py”, line 602, in parse_args config.from_dict(vars(opts)) File ...