Error using unicode data from TVDB

Trying to clear up some Seaquest episodes, and tvnamer fails on one episode called “Give Me Liberté”

####################
# Processing file: SeaQuest DSV - 1x06 - Treasures of the Tonga Trench.avi
# Detected series: SeaQuest DSV (season: 1, episode: 6)
####################
Old filename: SeaQuest DSV - 1x06 - Treasures of the Tonga Trench.avi
New filename: SeaQuest DSV - [S01E06] - Give Me Liberté.avi
Traceback (most recent call last):
  File "/usr/local/bin/tvnamer", line 9, in <module>
    load_entry_point('tvnamer==2.2.1', 'console_scripts', 'tvnamer')()
  File "/usr/local/lib/python2.7/dist-packages/tvnamer/main.py", line 413, in main
    tvnamer(paths = sorted(args))
  File "/usr/local/lib/python2.7/dist-packages/tvnamer/main.py", line 322, in tvnamer
    processFile(tvdb_instance, episode)
  File "/usr/local/lib/python2.7/dist-packages/tvnamer/main.py", line 213, in processFile
    doRenameFile(cnamer, newName)
  File "/usr/local/lib/python2.7/dist-packages/tvnamer/main.py", line 91, in doRenameFile
    cnamer.newName(newName, force = Config['overwrite_destination_on_rename'])
  File "/usr/local/lib/python2.7/dist-packages/tvnamer/utils.py", line 966, in newName
    if os.path.isfile(newpath):
  File "/usr/lib/python2.7/genericpath.py", line 29, in isfile
    st = os.stat(path)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 98: ordinal not in range(128)

[lortordermur]

I was able to reproduce the issue several times with German-localized anime titles using vanilla tvnamer (so, not inside my script). Kämpfer is especially interesting since the original title is a German word (with an umlaut), and such it applies to all dubs.

$ tvnamer --name Kämpfer *.mkv                                     
####################
# Starting tvnamer
# Found 3 episodes
####################
# Processing file: 01x01.mkv
Traceback (most recent call last):
  File "/usr/bin/tvnamer", line 4, in <module>
    main()
  File "/usr/share/tvnamer/main.py", line 418, in main
    tvnamer(paths = sorted(args))
  File "/usr/share/tvnamer/main.py", line 325, in tvnamer
    processFile(tvdb_instance, episode)
  File "/usr/share/tvnamer/main.py", line 167, in processFile
    p("# Detected series: %s (%s)" % (episode.seriesname, episode.number_string()))
  File "/usr/share/tvnamer/unicode_helper.py", line 34, in p
    new_args.append(x.encode(kw['encoding']))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 20: ordinal not in range(128)

However, Kampfer seems to redirect to the English dub of Kämpfer, and only that dub. Dutch, German, Portuguese and French are not offered at all.

$ tvnamer --name Kampfer *.mkv
####################
# Starting tvnamer
# Found 3 episodes
####################
# Processing file: 01x01.mkv
# Detected series: Kampfer (season: 1, episode: 1)
TVDB Search Results:
1 -> Kämpfer [en] # http://thetvdb.com/?tab=series&id=115991&lid=7 (default)
Automatically selecting only result
####################
Old filename: 01x01.mkv
New filename: Kämpfer - [01x01] - Destiny, the Chosen Ones.mkv
Rename?
([y]/n/a/q) q
Quitting

(As a translator I cannot withhold the fact that Kämpfer=fighter and Kampfer=camphor. Swapping “ä” with “a” even makes it an entirely different term, because “ä” is not just a “long a”–like in Hungarian–or some kind of accent, but a separate letter and phoneme contracted from an “ae” ligature back in the days of movable types.)

My filesystem encoding (Linux+ext4):

$ python -c "import sys; print sys.getfilesystemencoding()"
UTF-8

The Unicode issue does not seem to be on sides of the TVDB API, as typing the series title in the website’s search field works as expected. However it is potentially a very serious issue with domestic series in places where an extended Latin or a non-Latin alphabet or writing system is used. That means, quite all of Eurasia except the British Isles.

[dbr]

Closing old tickets - newer versions shouldn’t have this exact problem at least, as the entire unicode_helper.py module has been removed in current development version. If it’s still an issue, best to make new ticket