How to preprocess mask for multiclass segmentation?

Hello. Images and masks in PNG format. 8 Classes. How to preprocess masks for categorical_crossentropy loss right?

I tried to use (128, 128, 1) mask, setting each pixel of the mask with a number from 0 to 7, but got error: Error when checking target: expected softmax to have shape (128, 128, 8) but got array with shape (128, 128, 1)

Code for model creation: `BACKBONE = ‘resnet50’ preprocess_input = get_preprocessing(BACKBONE)

model = Unet(BACKBONE, input_shape=(128,128,3), encoder_weights=‘imagenet’, encoder_freeze=True, classes=8, activation=‘softmax’)

model.compile(‘Adam’, loss=‘categorical_crossentropy’, metrics=[iou_score])`

Thanks. Sorry for bad English.

[garelli]

I have the answer;

You need indeed to convert your RGB mask to a one-hot encoding image with shape (H,W,Channels) with Channels equals to the number of classes (containing the background).

Imagine you have an image/array (a mask) of shape (128,128,3). First you need to notice the unique elements which are corresponding to a label.

img.shape = (128,128,3)
list= []
for i in range(128):
    for j in range(128):
        list.append(img[i,j,:].tolist())

def uniq(lst):
    last = object()
    for item in lst:
        if item == last:
            continue
        yield item
        last = item

def sort_and_deduplicate(l):
    return list(uniq(sorted(l, reverse=True)))

palette = sort_and_deduplicate(list)
palette

[[128.0, 128.0, 0.0],
 [128.0, 0.0, 0.0],
 [0.0, 128.0, 0.0],
 [0.0, 0.0, 128.0],
 [0.0, 0.0, 0.0]]

palette is your current label pixels values in your RGB MASK (the unique elements)

Then, you can use a function like this to convert it to one-hot encoding:

one_hot_map = []
for colour in palette:
    class_map = tf.reduce_all(tf.equal(img2, colour), axis=-1)
    one_hot_map.append(class_map)
one_hot_map = tf.stack(one_hot_map, axis=-1)
one_hot_map = tf.cast(one_hot_map, tf.float32)

with tf.Session() as sess:
    print (sess.run(one_hot_map))

[[[0. 0. 0. 0. 1.]
  [0. 0. 0. 0. 1.]
  [0. 0. 0. 0. 1.]
  ...
  [0. 0. 0. 0. 1.]
  [0. 0. 0. 0. 1.]
  [0. 0. 0. 0. 1.]]

Here is for the processing of the masks (in this example you will have a (128,128,5) one-hot encoding mask.

And you can run

define model with output of N classes, where N > 1

model = Unet(‘resnet34’, classes=5, activation=‘softmax’)

for multiclass segmentation choose another loss and metric

model.compile(‘Adam’, loss=‘categorical_crossentropy’, metrics=[‘categorical_accuracy’])

Once your model is trained, the predict function will outputs a (128,128,5) mask with probability inside it.

Then you can use argmax function with numpy to convert it to (128,128,3) true image and visualize results.

Hope it will helps

[qubvel]

There is an example in examples/ dir now