Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Properly check for UTF8 console

See original GitHub issue
Steps to reproduce the problem:
  1. mitmproxy and mitmproxy --save-stream-file +/tmp/mitmdump
  2. select some event, enter
  3. scroll down
Traceback (most recent call last):
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/mitmproxy/tools/console/master.py", line 216, in run
    self.loop.run()
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/main_loop.py", line 286, in run
    self._run()
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/main_loop.py", line 384, in _run
    self.event_loop.run()
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/main_loop.py", line 788, in run
    self._loop()
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/main_loop.py", line 816, in _loop
    self._entering_idle()
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/main_loop.py", line 777, in _entering_idle
    callback()
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/main_loop.py", line 572, in entering_idle
    self.draw_screen()
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/main_loop.py", line 586, in draw_screen
    canvas = self._topmost_widget.render(self.screen_size, focus=True)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/container.py", line 1086, in render
    focus and self.focus_part == 'body')
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/decoration.py", line 226, in render
    canv = self._original_widget.render(size, focus=focus)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/container.py", line 1086, in render
    focus and self.focus_part == 'body')
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/container.py", line 1086, in render
    focus and self.focus_part == 'body')
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/widget.py", line 1765, in render
    canv = get_delegate(self).render(size, focus=focus)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/container.py", line 1086, in render
    focus and self.focus_part == 'body')
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/listbox.py", line 489, in render
    focus_canvas = focus_widget.render((maxcol,), focus=focus)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/widget.py", line 144, in cached_render
    canv = fn(self, size, focus=focus)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/widget.py", line 1004, in render
    return apply_text_layout(text, attr, trans, maxcol)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/canvas.py", line 1315, in apply_text_layout
    return TextCanvas(t, a, c, maxcol=maxcol)
  File "/usr/local/Cellar/mitmproxy/3.0.0/libexec/lib/python3.6/site-packages/urwid/canvas.py", line 358, in __init__
    raise CanvasError("Canvas text is wider than the maxcol specified \n%r\n%r\n%r"%(maxcol,widths,text))
urwid.canvas.CanvasError: Canvas text is wider than the maxcol specified
153
[153, 153, 154, 125]
[b'[REDACTED NAME]:        ....$.......?.E=?[LONG REDACTED GLIBBERISH]"]

pasted_image_22_02_18_18_36

System information
$ mitmproxy --version
Mitmproxy: 3.0.0
Python:    3.6.4
OpenSSL:   OpenSSL 1.0.2n  7 Dec 2017
Platform:  Darwin-13.4.0-x86_64-i386-64bit

(via Brew, installed earlier today, so should be the latest version)

From the second run with dumping to file, this seems to be the binary form-data where it crashed. The request is to an API’s upload function.

2018-02-23 08:00:00
--Boundary+1DAA87079F6BACB5
Content-Disposition: form-data; name="[REDACTED NAME]"; filename="file"
Content-Type: application/octet-stream

^A^@^@^@[REDACTED GLIBBERISH]

As I am not certain about the contents of the recorded data, so I won’t post the dump public. However, I have keep it for now.

Also might be another occurrence of #1796.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:16 (6 by maintainers)

github_iconTop GitHub Comments

3reactions
yssourcecommented, Oct 15, 2020

mitmproxy --version

Mitmproxy: 5.2
Python:    3.7.7+
OpenSSL:   OpenSSL 1.1.1g  21 Apr 2020
Platform:  Linux-4.19.150-1-MANJARO-x86_64-with-arch-Manjaro-Linux

locale -a

C
en_US.utf8
ja_JP.utf8
POSIX
zh_CN.utf8

tput cols

240

Traceback (most recent call last):
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/mitmproxy/master.py", line 86, in run_loop
    loop()
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/main_loop.py", line 287, in run
    self._run()
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/main_loop.py", line 385, in _run
    self.event_loop.run()
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/main_loop.py", line 1494, in run
    reraise(*exc_info)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/compat.py", line 58, in reraise
    raise value
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/main_loop.py", line 1454, in faux_idle_callback
    callback()
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/main_loop.py", line 574, in entering_idle
    self.draw_screen()
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/main_loop.py", line 588, in draw_screen
    canvas = self._topmost_widget.render(self.screen_size, focus=True)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/widget.py", line 145, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/container.py", line 1086, in render
    focus and self.focus_part == 'body')
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/widget.py", line 145, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/decoration.py", line 226, in render
    canv = self._original_widget.render(size, focus=focus)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/widget.py", line 145, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/container.py", line 1086, in render
    focus and self.focus_part == 'body')
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/widget.py", line 145, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/listbox.py", line 445, in render
    focus_canvas = focus_widget.render((maxcol,), focus=focus)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/widget.py", line 145, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/widget.py", line 1761, in render
    canv = get_delegate(self).render(size, focus=focus)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/widget.py", line 145, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/container.py", line 2125, in render
    focus = focus and self.focus_position == i)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/widget.py", line 145, in cached_render
    canv = fn(self, size, focus=focus)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/mitmproxy/tools/console/common.py", line 207, in render
    return urwid.TextCanvas([c_text.encode()], [c_attr], maxcol=width)
  File "/home/jimmy/.pyenv/versions/3.7-dev/lib/python3.7/site-packages/urwid/canvas.py", line 358, in __init__
    raise CanvasError("Canvas text is wider than the maxcol specified \n%r\n%r\n%r"%(maxcol,widths,text))
urwid.canvas.CanvasError: Canvas text is wider than the maxcol specified
21
[26]
[b'\xe2\x80\xa6": [Errno 101] \xe7\xbd\x91\xe7\xbb\x9c\xe4\xb8\x8d\xe5\x8f\xaf\xe8\xbe\xbe']

mitmproxy has crashed!
Please lodge a bug report at:
	https://github.com/mitmproxy/mitmproxy
2reactions
Jessonsotoventuracommented, Jan 3, 2020

Thanks for the instructions - I was able to replicate it. @mhils I can take a look at this

Read more comments on GitHub >

github_iconTop Results From Across the Web

Properly print utf8 characters in windows console
UTF-8 text can be printed directly to the console by setting the console output codepage to CP_UTF8, ... To check the UTF-8 support,...
Read more >
Unicode and UTF-8 Output Text Buffer - Windows Command ...
Windows Command-Line: Unicode and UTF-8 Output Text Buffer ... and measure the Console's performance, memory footprint, power consumption, ...
Read more >
Unicode characters are not appearing properly in console and ...
For console-related issues first set the encoding type in the Web Browser (in Internet Explorer: View->Encoding->UTF-8). If inputting data in ...
Read more >
Is there a Linux command to find out if a file is UTF-8?
You can determine the file encoding with the following command: file -bi filename ... You can verify if a file happens to pass...
Read more >
UTF-8 characters are not printed correctly on the console
I found a solution! I simply had to convert the files to UTF-8 , they where in ISO-8859 . To see the file's...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Docs Improvements

Next page

Traceback during installation in Termux