ImageSegmentaion

Task Module: Semantic Segmentation (2D)

Task Description

• Input: one image or more, each of them contains one or more objects
• Output: for each input image, each pixel on the image is classified which object or class it belongs to.

Evaluation Metrics: mIoU (mean intersection over union)

1. For each segmentation generated, the foreground IoU is calculated, which is between 0~1
2. Then IoU is computed on a per-class basis and then averaged as mIoU

Benchmark Datasets

1. PASCAL Visual Object Classification (PASCAL VOC) dataset
2. Common Objects in COntext (COCO) dataset
3. Cityscapes

Reason

Semantic Segmentation is one of the most important research areas and applications in Computer Vision, such as autonomous driving, indoor navigation. This module will provide easy access to people or group who needs to perform semantic segmentation in their work with limited Computer Vision or Machine Learning background.

Solution

Some add-ons on current AutoKeras to be considered:

1. Add unpooling layer into search space, and apply restrictions during search (e.g. alignment of pooling/unpooling layers).
2. Consider using some popular architectures as initializations (e.g. UNet, DeepLab).
3. Consider adding complexity constraint (e.g. depth, width) in searching metrics. Currently the accuracy is the only metrics for the model selection.

[bhack]

In future, we plan to incorporate more operations and optimizations into our search space, and apply it to more mobile vision tasks such as semantic segmentation.

https://ai.googleblog.com/2018/08/mnasnet-towards-automating-design-of.html

[bhack]

Searching for Efficient Multi-Scale Architectures for Dense Image Prediction code just released https://github.com/tensorflow/models/pull/5430