[FEATURE]: Enable masking of unlabeled portions of imagery during model training

This was a topic raised by an attendee at a workshop at FOSS4G International 2019 and is something we should consider implementing.

At the moment, we assume that all pixels in the input imagery are labeled when creating pixel masks and training models. Any unlabeled portion is assumed to have no objects of interest in it. There are some use cases where only a portion of an input image is labeled, and the user only wants the model to learn based on the labeled portion. There are a few ways we could support this:

1. Enable selective tiling where unlabeled portions are filled with nodata values;
2. Enable selective masking where unlabeled portions of masks are filled with NaN or something like that;
3. Find a way to have loss functions have NA values in specific regions of the images so those get ignored during model training.

.

1. is good though if there’s some bias in the labels near the edges of the labeled area, it could introduce bias in the model (it may learn that near nodata values some targets are more/less likely to occur).
2. Is good but would be hard;
3. Is good but would be really hard, particularly as we’d have to re-write every loss function anytime we added a new one. To some degree we’d have to do the same for 2., though it would just be with how to handle NaN inputs.

I’m inclined to go with 1. or 2. if we implement this. We’d definitely need tests implemented to make sure it works.

[rbavery]

Got it, should have made it clear that what I’m discussing is a separate problem from filling nodata areas outside of an unlabeled boundary. To be more clear, I was discussing how to handle the case where labels are present but no valid image values exist (either due to edge effects or cloud masking). I’ve made a separate issue here: https://github.com/CosmiQ/solaris/issues/328

[nrweir]

I don’t think this actually fully solves the intention of this request - specifically, to find a way to mask out areas from contributing to the loss function. So I’ll leave it open for now.