KeyError: Options in training stage for recipes

🐛 Bug

When I tried to run a recipe (SuDORMRFNET and SuDORMRFImprovedNet), I always end up running into this problem in stage 2 like the following.

Stage 2: Training Traceback (most recent call last): File “/mnt/c/Project/asteroid/egs/librimix/SuDORMRFNet/train.py”, line 140, in <module> arg_dic, plain_args = parse_args_as_dict(parser, return_plain_args=True) File “/home/cchoi1022/cd/envs/asteroid/lib/python3.10/site-packages/asteroid/utils/parser_utils.py”, line 135, in parse_args_as_dict args_dic[“main_args”] = args_dic[“optional arguments”] KeyError: ‘optional arguments’

To Reproduce

cd /asteroid/egs/librimix/SuDORMRFImprovedNet . ./run.sh

or python train.py

Expected behavior

The training commences

Environment

Package versions

Run asteroid-versions and paste the output here: Traceback (most recent call last): File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 584, in _build_master ws.require(requires) File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 901, in require needed = self.resolve(parse_requirements(requirements)) File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 792, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.ContextualVersionConflict: (PyYAML 5.3.1 (/usr/lib/python3/dist-packages), Requirement.parse(‘PyYAML>=5.4’), {‘pytorch-lightning’})

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/home/cchoi1022/.local/bin/asteroid-versions”, line 33, in <module> sys.exit(load_entry_point(‘asteroid’, ‘console_scripts’, ‘asteroid-versions’)()) File “/home/cchoi1022/.local/bin/asteroid-versions”, line 25, in importlib_load_entry_point return next(matches).load() File “/usr/lib/python3.8/importlib/metadata.py”, line 77, in load module = import_module(match.group(‘module’)) File “/usr/lib/python3.8/importlib/init.py”, line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File “<frozen importlib._bootstrap>”, line 1014, in _gcd_import File “<frozen importlib._bootstrap>”, line 991, in _find_and_load File “<frozen importlib._bootstrap>”, line 975, in _find_and_load_unlocked File “<frozen importlib._bootstrap>”, line 671, in _load_unlocked File “<frozen importlib._bootstrap_external>”, line 848, in exec_module File “<frozen importlib._bootstrap>”, line 219, in _call_with_frames_removed File “/mnt/c/Project/asteroid/asteroid/scripts/asteroid_versions.py”, line 5, in <module> import pytorch_lightning as pl File “/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/init.py”, line 30, in <module> from pytorch_lightning.callbacks import Callback # noqa: E402 File “/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/callbacks/init.py”, line 14, in <module> from pytorch_lightning.callbacks.base import Callback File “/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/callbacks/base.py”, line 25, in <module> from pytorch_lightning.utilities.types import STEP_OUTPUT File “/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/init.py”, line 18, in <module> from pytorch_lightning.utilities.apply_func import move_data_to_device # noqa: F401 File “/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/apply_func.py”, line 29, in <module> from pytorch_lightning.utilities.imports import _compare_version, _TORCHTEXT_LEGACY File “/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/imports.py”, line 22, in <module> import pkg_resources File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 3254, in <module> def _initialize_master_working_set(): File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 3237, in _call_aside f(*args, **kwargs) File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 3266, in _initialize_master_working_set working_set = WorkingSet._build_master() File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 586, in _build_master return cls._build_from_requirements(requires) File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 599, in _build_from_requirements dists = ws.resolve(reqs, Environment()) File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 792, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.ContextualVersionConflict: (PyYAML 5.3.1 (/usr/lib/python3/dist-packages), Requirement.parse(‘PyYAML>=5.4’), {‘pytorch-lightning’}) (asteroid) cchoi1022@DESKTOP-Q470KUQ:/mnt/c/Project/asteroid/egs/librimix/SuDORMRFImprovedNet$ pyYAML pyYAML: command not found (asteroid) cchoi1022@DESKTOP-Q470KUQ:/mnt/c/Project/asteroid/egs/librimix/SuDORMRFImprovedNet$ pip install --upgrade pyyaml Requirement already satisfied: pyyaml in /home/cchoi1022/cd/envs/asteroid/lib/python3.10/site-packages (5.4.1) Collecting pyyaml Using cached PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB) Installing collected packages: pyyaml Attempting uninstall: pyyaml Found existing installation: PyYAML 5.4.1 Uninstalling PyYAML-5.4.1: Successfully uninstalled PyYAML-5.4.1 Successfully installed pyyaml-6.0 (asteroid) cchoi1022@DESKTOP-Q470KUQ:/mnt/c/Project/asteroid/egs/librimix/SuDORMRFImprovedNet$ asteroid-versions Traceback (most recent call last): File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 584, in _build_master ws.require(requires) File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 901, in require needed = self.resolve(parse_requirements(requirements)) File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 792, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.ContextualVersionConflict: (PyYAML 5.3.1 (/usr/lib/python3/dist-packages), Requirement.parse(‘PyYAML>=5.4’), {‘pytorch-lightning’})

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “/home/cchoi1022/.local/bin/asteroid-versions”, line 33, in <module> sys.exit(load_entry_point(‘asteroid’, ‘console_scripts’, ‘asteroid-versions’)()) File “/home/cchoi1022/.local/bin/asteroid-versions”, line 25, in importlib_load_entry_point return next(matches).load() File “/usr/lib/python3.8/importlib/metadata.py”, line 77, in load module = import_module(match.group(‘module’)) File “/usr/lib/python3.8/importlib/init.py”, line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File “<frozen importlib._bootstrap>”, line 1014, in _gcd_import File “<frozen importlib._bootstrap>”, line 991, in _find_and_load File “<frozen importlib._bootstrap>”, line 975, in _find_and_load_unlocked File “<frozen importlib._bootstrap>”, line 671, in _load_unlocked File “<frozen importlib._bootstrap_external>”, line 848, in exec_module File “<frozen importlib._bootstrap>”, line 219, in _call_with_frames_removed File “/mnt/c/Project/asteroid/asteroid/scripts/asteroid_versions.py”, line 5, in <module> import pytorch_lightning as pl File “/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/init.py”, line 30, in <module> from pytorch_lightning.callbacks import Callback # noqa: E402 File “/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/callbacks/init.py”, line 14, in <module> from pytorch_lightning.callbacks.base import Callback File “/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/callbacks/base.py”, line 25, in <module> from pytorch_lightning.utilities.types import STEP_OUTPUT File “/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/init.py”, line 18, in <module> from pytorch_lightning.utilities.apply_func import move_data_to_device # noqa: F401 File “/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/apply_func.py”, line 29, in <module> from pytorch_lightning.utilities.imports import _compare_version, _TORCHTEXT_LEGACY File “/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/imports.py”, line 22, in <module> import pkg_resources File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 3254, in <module> def _initialize_master_working_set(): File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 3237, in _call_aside f(*args, **kwargs) File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 3266, in _initialize_master_working_set working_set = WorkingSet._build_master() File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 586, in _build_master return cls._build_from_requirements(requires) File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 599, in _build_from_requirements dists = ws.resolve(reqs, Environment()) File “/usr/lib/python3/dist-packages/pkg_resources/init.py”, line 792, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.ContextualVersionConflict: (PyYAML 5.3.1 (/usr/lib/python3/dist-packages), Requirement.parse(‘PyYAML>=5.4’), {‘pytorch-lightning’})

Alternatively, if you cannot install Asteroid or have an old version that doesn’t have the asteroid-versions script, please output the output of:

pip freeze | egrep -i 'pytorch|torch|asteroid'

Additional info

Additional info (environment, custom script, etc…)

[mpariente]

Sorry, I don’t have any bandwidth to debug this. My tip : have a look at the other recipes and look at what’s different in the argument parsing, you’ll probably find our error

[joeoct93]

I tried looking online for solutions, but could not find anything. Apparently this KeyError is telling me that “optional arguments” is not in the dictionary. But I simply installed asteroid, I don’t understand why ‘optional arguments’ is not in the dictionary.