Way to use weighted loss in config

I use configs to train my nets and now I’ve faced imbalanced classification problem. It could be solved by training classifier with weighted loss and pytorch losses support weighting out of the box.

But I couldn’t use them because they accept only tensor input. If I provide a list of desired weights via config it get parsed to regular python list, so loss constructor fails.

To solve this I’ve written own wrapper class that constructs tensor and passes it to loss:

class WrappedCrossEntropyLoss(nn.CrossEntropyLoss):
    '''Implements standard CrossEntropyLoss, where weight param can be list, not tensor'''

    def __init__(self, weights: Optional[List] = None):
        weights_tensor = torch.tensor(weights) if weights is not None else None
        super().__init__(weights_tensor)

Then regular config works fine:

    criterion_params:
        criterion: WrappedCrossEntropyLoss
        weights: [0.2491, 1.0]

Is is intentional behaviour and could catalyst somehow cast such arguments automatically?

I guess it could be solved by enhancing config parsing algorithm. In case you are interested in solving this I could create some PR.

[v-goncharenko]

AFAIK, PyTorch always accepted only tensor as an input for loss classes (any other type just fails), so I believe they won’t change this (although I will search their issues for details). So PR to PyTorch doesn’t look like a probable solution.

So for now answer to my initial question is No - one couldn’t use weights param of nn.CrossEntropyLoss directly from config (without some wrapper class).

Regarding designing config syntax - I’ll contact you =)

[Scitator]

Config-way of object creation is no the pure PyTorch way 😃 So, you need to write your wrappers.

btw, speaking of theweights example, I think it should work naturally thanks to python dynamic cast list -> tensor during the __init__ in the class. If no, it looks like we need to PR to the PyTorch 😃

another opinion, that I see - add eval option to the Catalyst Config API, for example

   criterion_params:
        criterion: WrappedCrossEntropyLoss
        weights: "torch.tensor([0.2491, 1.0])"

or

   criterion_params:
        criterion: WrappedCrossEntropyLoss
        weights: "[0.2491, 1.0])":torch.tensor

Nevertheless, we need to design it properly first. If you would like to make such contribution - feel free to write me in slack 😉