Exception during exception handling

Hey,

I have some issues with the exception handling in sacred: Consider the slightly modified “hello world” example from the docs which throws an exception:

main.py:

from sacred import Experiment

ex = Experiment('hello_config')

@ex.config
def my_config():
    recipient = "world"
    message = "Hello %s!" % recipient

@ex.automain
def my_main(message):
    raise ValueError("Some Error")

Now, python main.py produces the following output:

WARNING - hello_config - No observers have been added to this run
INFO - hello_config - Running command 'my_main'
INFO - hello_config - Started
ERROR - hello_config - Failed after 0:00:00!
Traceback (most recent call last):
  File "/home/markus/test/sacred_test/env/lib64/python3.9/site-packages/sacred/experiment.py", line 312, in run_commandline
    return self.run(
  File "/home/markus/test/sacred_test/env/lib64/python3.9/site-packages/sacred/experiment.py", line 276, in run
    run()
  File "/home/markus/test/sacred_test/env/lib64/python3.9/site-packages/sacred/run.py", line 238, in __call__
    self.result = self.main_function(*args)
  File "/home/markus/test/sacred_test/env/lib64/python3.9/site-packages/sacred/config/captured_function.py", line 42, in captured_function
    result = wrapped(*args, **kwargs)
  File "/home/markus/test/sacred_test/main.py", line 12, in my_main
    raise ValueError("Some Error")
ValueError: Some Error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/markus/test/sacred_test/main.py", line 11, in <module>
    def my_main(message):
  File "/home/markus/test/sacred_test/env/lib64/python3.9/site-packages/sacred/experiment.py", line 190, in automain
    self.run_commandline()
  File "/home/markus/test/sacred_test/env/lib64/python3.9/site-packages/sacred/experiment.py", line 347, in run_commandline
    print_filtered_stacktrace()
  File "/home/markus/test/sacred_test/env/lib64/python3.9/site-packages/sacred/utils.py", line 493, in print_filtered_stacktrace
    print(format_filtered_stacktrace(filter_traceback), file=sys.stderr)
  File "/home/markus/test/sacred_test/env/lib64/python3.9/site-packages/sacred/utils.py", line 528, in format_filtered_stacktrace
    return "".join(filtered_traceback_format(tb_exception))
  File "/home/markus/test/sacred_test/env/lib64/python3.9/site-packages/sacred/utils.py", line 568, in filtered_traceback_format
    current_tb = tb_exception.exc_traceback
AttributeError: 'TracebackException' object has no attribute 'exc_traceback'

i.e. there is another exception during the handling of the ValueError.

This was produced on a fresh virtual environment with Python 3.9.7 and Sacred 0.8.2. The behavior is reproducible across several machines and was also observed by a coworker. It is also mentioned here: https://stackoverflow.com/questions/67014839/attributeerror-tracebackexception-object-has-no-attribute-exc-trackback-err and here: https://github.com/ray-project/ray/issues/5181 This issue seems to be relatively new, but I cannot track down, what versions/changes introduced this behavior.

This is a major issue for us because we intentionally raise exceptions to log and schedule restarts of the experiments.

I would be grateful for some help!

Best, Markus

[Rocamonde]

Same exception, same problem. This does not seem to be fixed.

[thequilo]

Yes, I think that that commit fixed the exception handling for Python 3.9. It could be time for a new release on pypi with the fixes for Python 3.9.