question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Cannot instantiate augmentations with tuple parameters from Hydra config

See original GitHub issue

🐛 Bug

I’d like to use Hydra to configure and instantiate augmentations. It works nicely, unless I try to use a tuple as a parameter value. Hydra translates lists and tuples to an internal type that can be casted to a tuple, but Albumentations checks the parameter types and won’t try to cast it if it’s not a list.

To Reproduce

It’s convenient to define augmentations in a Hydra configuration like this:

augmentations:
- _target_: albumentations.HorizontalFlip
  p: 0.5
- _target_: albumentations.HueSaturationValue
  sat_shift_limit:
  - -10
  - 20

One can then instantiate the augmentations like this:

from omegaconf import DictConfig
import hydra
from hydra.utils import instantiate

@hydra.main(config_path='conf', config_name='config')
def my_app(cfg: DictConfig) -> None:
    augmentations = [instantiate(a) for a in cfg.augmentations]

if __name__ == "__main__":
    my_app()

If you run the above code, it will correctly instantiate the HorizontalFlip augmentation, but fails to instantiate the HueSaturationValue. The reason is that instantiate() translates the sat_shift_limit value [-10, 20] to an omegaconf.ListConfig. It would be possible to cast omegaconf.ListConfig to a tuple, but to_tuple() checks that the type is either a list or a tuple before casting. It’s not, so instead of casting it throws an exception:

ValueError: Argument param must be either scalar (int, float) or tuple

Expected behavior

to_tuple() should cast the omegaconf.ListConfig to a tuple, so that instantiate() would instantiate the HueSaturationValue transform.

Environment

  • Albumentations version (e.g., 0.1.8): 0.5.2
  • Python version (e.g., 3.7): 3.8.3
  • OS (e.g., Linux): Linux
  • How you installed albumentations (conda, pip, source): conda

Additional context

I understand that it’s impossible to check for omegaconf.ListConfig and all other possible cases that can be casted to a tuple. Actually it’s common to use “duck typing” in Python, meaning that anything that acts like a list can be provided in place of a list, so it’s problematic to check the exact type of parameters.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
BloodAxecommented, Apr 28, 2021

It seems hydra (Tested on 1.1) instantiate has additional parameter _convert_:

   _convert_: Conversion strategy
        none    : Passed objects are DictConfig and ListConfig, default
        partial : Passed objects are converted to dict and list, with
                  the exception of Structured Configs (and their fields).
        all     : Passed objects are dicts, lists and primitives without
                  a trace of OmegaConf containers

So adding _convert_ should do the job:

augmentations:
- _target_: albumentations.HorizontalFlip
  p: 0.5
- _target_: albumentations.HueSaturationValue
  _convert_: all
  sat_shift_limit:
  - [10, 20]
1reaction
Dipetcommented, Apr 28, 2021

It looks like we can replace check isinstance(param, (list, tuple)) with isinstance(param, Sequence) and everything will work fine. Sequence - is a standard type from typing module

Read more comments on GitHub >

github_iconTop Results From Across the Web

detectron2.config
Recursively instantiate objects defined in dictionaries by “_target_” and arguments. Parameters. cfg – a dict-like object with “_target_” that defines the ...
Read more >
NeMo Models - NVIDIA Documentation Center
NeMo uses Hydra for configuring both NeMo models and the PyTorch Lightning Trainer. ... As an example, we can instantiate QuartzNet with the...
Read more >
geo-deep-learning/train_segmentation.py at develop - GitHub
Function to retrieve number of patches, either from config file or directly from csv listing all created patches. @param patches_path: (str) Path to...
Read more >
Algorithm Configuration: A Hands-on Tutorial | AutoML
An instance of the algorithm configuration problem is a 5-tuple. (A,Θ,D, κ, m) where: A is a parameterized algorithm;. Θ is the parameter...
Read more >
how to do tuple unpacking in python Code Example - Code Grepper
#tuple unpacking allows us to store each tuple element in a variable. 4. #syntax - vars = tuple. 5. """NOTE: no of vars...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found