running ciphey -f against file with unknown text encoding causes system hang, process gets killed on Ubuntu 20.04

Describe the bug running ciphey -f against a file with unknown text encoding/encryption causes a system hang and out of memory on Ubuntu 20.04 , the ciphey process prints a Killed message to the terminal

Other applications (such as Firefox) and the desktop become unresponsive for a time, on Ubuntu usually an indicator that a process is consuming too much memory thereby causing system instability.

To Reproduce Steps to reproduce the behavior:

1. encoded text from https://pastebin.com/raw/rpm0YKdg
2. install, run in virtualenv
3. ciphey -f rpm0YKdg.txt

Verbose Output

$ ciphey -vvv
2020-08-11 22:24:16.605 | DEBUG    | ciphey.iface._config:update_log_level:176 - Verbosity set to level 3 (TRACE)
2020-08-11 22:24:16.605 | DEBUG    | ciphey.ciphey:main:214 - Opened config file at /home/user/.config/ciphey/config.yml
2020-08-11 22:24:16.605 | TRACE    | ciphey.ciphey:main:215 - Got cmdline args {'verbose': 3, 'text_stdin': None, 'text': None, 'quiet': None, 'greppable': None, 'checker': None, 'config': None, 'wordlist': None, 'param': (), 'list_params': None, 'searcher': None, 'bytes_input': None, 'bytes_output': None, 'default_dist': None, 'module': (), 'appdirs': False, 'file': None}
2020-08-11 22:24:16.605 | DEBUG    | ciphey.iface._config:load_modules:186 - Loaded modules ['regex', 'regexlist', 'brandon', 'jsonchecker', 'ezcheck', 'caesar', 'vigenere', 'morsecode', 'base16', 'base32', 'base64', 'base85', 'ascii85', 'utf8', 'reverse', 'octal', 'binary', 'hex', 'atbash', 'cipheydists', 'json', 'csv', 'ausearch']
2020-08-11 22:24:16.606 | TRACE    | ciphey.iface._config:get_resource:196 - Loading resource cipheydists::list::flags of type None
2020-08-11 22:24:16.606 | TRACE    | ciphey.basemods.Resources.cipheydists:getResource:30 - Loading cipheydists resource list::flags
2020-08-11 22:24:16.606 | TRACE    | ciphey.basemods.Checkers.regex:__init__:49 - There are 4 regexes
2020-08-11 22:24:16.606 | TRACE    | ciphey.iface._config:get_resource:196 - Loading resource cipheydists::brandon::english of type None
2020-08-11 22:24:16.606 | TRACE    | ciphey.basemods.Resources.cipheydists:getResource:30 - Loading cipheydists resource brandon::english
2020-08-11 22:24:16.606 | TRACE    | ciphey.iface._config:get_resource:196 - Loading resource cipheydists::list::english1000 of type None
2020-08-11 22:24:16.607 | TRACE    | ciphey.basemods.Resources.cipheydists:getResource:30 - Loading cipheydists resource list::english1000
2020-08-11 22:24:16.607 | TRACE    | ciphey.iface._config:get_resource:196 - Loading resource cipheydists::list::english of type None
2020-08-11 22:24:16.607 | TRACE    | ciphey.basemods.Resources.cipheydists:getResource:30 - Loading cipheydists resource list::english
2020-08-11 22:24:16.638 | TRACE    | ciphey.iface._config:get_resource:196 - Loading resource cipheydists::list::englishStopWords of type None
2020-08-11 22:24:16.638 | TRACE    | ciphey.basemods.Resources.cipheydists:getResource:30 - Loading cipheydists resource list::englishStopWords
2020-08-11 22:24:16.640 | DEBUG    | ciphey.iface._config:update_log_level:176 - Verbosity set to level 3 (TRACE)
2020-08-11 22:24:16.640 | TRACE    | ciphey.ciphey:main:242 - Command line opts: {'verbose': 3, 'text_stdin': None, 'text': None, 'quiet': None, 'greppable': None, 'checker': None, 'config': None, 'wordlist': None, 'param': (), 'list_params': None, 'searcher': None, 'bytes_input': None, 'bytes_output': None, 'default_dist': None, 'module': (), 'appdirs': False, 'file': None}
2020-08-11 22:24:16.640 | TRACE    | ciphey.ciphey:main:243 - Config finalised: {'verbosity': 3, 'searcher': 'ausearch', 'params': {'regexlist': {'resource': ['cipheydists::list::flags']}, 'brandon': {'top1000': 'cipheydists::list::english1000', 'wordlist': 'cipheydists::list::english', 'stopwords': 'cipheydists::list::englishStopWords', 'threshold': 0.45, 'phases': 'cipheydists::brandon::english'}, 'ausearch': {'invert_priority': 'False', 'disable_priority': 'True'}}, 'format': {'in': 'str', 'out': 'str'}, 'modules': [], 'checker': 'ezcheck', 'default_dist': 'cipheydists::dist::english', 'timeout': None}
Error. No inputs were given to Ciphey. [old red]

config.yml is empty

Expected behavior ciphey process doesn’t hang system or get killed

Screenshots

Versions:

• OS/Distro: Ubuntu 20.04
• Python version: 3.8.2, virtualenv 20.0.30 from /usr/local/lib/python3.8/dist-packages/virtualenv/init.py
• Ciphey versions: ciphey 5.1.0 cipheycore 0.2.9 cipheydists 0.3.18

p.s. The text input is expected to be malicious payload of some kind (possibly PE exe format). If anyone knows what the encoded text is and how to decode, please let me know.

[Cyclic3]

Your ciphertext is clearly base64 encoded, but I cannot work out what format the bytes are in. It definitely isn’t machine code (as you might expect from a PE binary), and both binwalk and file have no idea what it could be. It may be a lot faster to look through the context for this data in the PE binary, perhaps you can uncover how it was encrypted there.

Ciphey succeeds in decoding it as base64, but then gets stuck trying to crack such a huge ciphertext (probably on vigenere). Whilst hanging is rather expected, it should definitely not cause an OOM condition on your system! I’ll see if I can track down the memory-hogging function, and fix this.

However, be aware that this fix may not actually get ciphey to decode your ciphertext. Ciphey only currently supports a subset of all ciphers and encodings, which is currently:

• caesar
• vigenere
• morsecode,
• base2/8/16/32/64/85 and ascii85
• utf8
• reverse
• atbash

With support for single and multi byte xor coming very soon. If the ciphertext is not made with one of these, then the chance of ciphey cracking it is very low.

Also, bear in mind that ciphey only stops when it finds something that looks like plaintext (currently english, json and CTF flags). If your result is not in this format, ciphey will just keep going ad infinitum. You can set up a regex if you have a crib (a known piece of the plaintext), or write a custom checker using our API if it is not in one of these forms.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.97. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.