ERROR: Trying auto compress example with GridSearch gets ValueError: Unsupported compression algorithm: l.

Describe the issue:

Trying to run examples/model_compress/auto_compress/torch/auto_compress_torch.py with GridSearch but failed. The modified auto_compress_torch.py is here:

# Copyright (c) Microsoft Corporation.
# Licensed under the MIT license.

from pathlib import Path

from nni.algorithms.compression.pytorch.auto_compress import AutoCompressionExperiment, AutoCompressionSearchSpaceGenerator

from auto_compress_module import AutoCompressionModule

generator = AutoCompressionSearchSpaceGenerator()

# Add the config for testing

generator.add_config('level', [
    {
        "sparsity": {
            "_type": "choice",
            "_value": [0.01, 0.99]
        },
        'op_types': ['default']
    }
])


search_space = generator.dumps()

experiment = AutoCompressionExperiment(AutoCompressionModule, 'local')
experiment.config.experiment_name = 'auto compression torch example'
experiment.config.trial_concurrency = 1
experiment.config.max_trial_number = 10
experiment.config.search_space = search_space
experiment.config.trial_code_directory = Path(__file__).parent

# Use the Grid Search tuner
experiment.config.tuner.name = 'GridSearch'

experiment.config.training_service.use_active_gpu = True

experiment.run(8088)

Environment:

• NNI version: 2.4
• Training service (local|remote|pai|aml|etc): local
• Client OS: Ubuntu 18.04
• Server OS (for remote mode only):
• Python version: Anaconda 3 (Python 3.8.5)
• PyTorch/TensorFlow version: PyTorch 1.8.1
• Is conda/virtualenv/venv used?: Yes
• Is running in Docker?: No

Configuration:

• Experiment config (remember to remove secrets!):
• Search space:

Log message:

• nnimanager.log:
• dispatcher.log:
• nnictl stdout and stderr:

The stderr of trials shows:

Traceback (most recent call last):
  File "/home/xxx/anaconda3/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/xxxxx/anaconda3/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/xxxxx/anaconda3/lib/python3.8/site-packages/nni/algorithms/compression/pytorch/auto_compress/trial_entry.py", line 25, in <module>
    AutoCompressionEngine.trial_execute_compress(module_name)
  File "/home/xxxxx/anaconda3/lib/python3.8/site-packages/nni/algorithms/compression/pytorch/auto_compress/auto_compress_engine.py", line 161, in trial_execute_compress
    compressed_model = cls._compress(compressor_type, algorithm_name, model, config_list, optimizer_factory,
  File "/home/xxxxx/anaconda3/lib/python3.8/site-packages/nni/algorithms/compression/pytorch/auto_compress/auto_compress_engine.py", line 138, in _compress
    compressed_model = func_dict[compressor_type](algorithm_name, model, config_list, optimizer_factory, criterion, sparsifying_trainer,
  File "/home/xxxxx/anaconda3/lib/python3.8/site-packages/nni/algorithms/compression/pytorch/auto_compress/auto_compress_engine.py", line 93, in __compress_pruning
    raise ValueError('Unsupported compression algorithm: {}.'.format(algorithm_name))
ValueError: Unsupported compression algorithm: l.

How to reproduce it?:

Run above modified code will reproduce it.

Go to the file location of the modified code. For example: $ cd /home/xxxxx/nni/examples/model_compress/auto_compress/torch/ $ python auto_compress_torch.py

I found the “l” in “ValueError: Unsupported compression algorithm: l.” is the first letter of level. That’s because nni/algorithms/hpo/gridsearch_tuner.py didn’t prepare parameters properly in the following code

        for val in values:
            for config in rest_para:
                config[key] = val
                ret_para.append(copy.deepcopy(config))

In the FOR LOOP when the values store the string “level” (from auto_compress_torch.py), each letter of “leave” becomes val. Then we see the “l” in “ValueError: Unsupported compression algorithm: l.”

The workaround is to do the following changes:

        if isinstance(values, str):
            values = [values]

        for val in values:
            for config in rest_para:
                config[key] = val
                ret_para.append(copy.deepcopy(config))

[J-shang]

The main problem is that the way Grid Search supported nested search space is different from the above tuners. We will fix this in the future.

[J-shang]

Hello, @shcchen auto compress only supports TPE Tuner, Random Search Tuner, Anneal Tuner, Evolution Tuner right now, sorry for confusing you, we will update the doc.