Add FCN Model for Segmentation

Short Description A fresh issue to discuss the implementation of the Fully Convolutional Network for Semantic Segmentation paper Last discussed here in #1099 As mentioned in Roadmap for Q4 2022

Papers Fully Convolutional Network for Semantic Segmentation - 34479 citations, 222 mentions (from arXiv scite_ tool)

Results from paper for task : Semantic Segmentation

Model Backbone	MeanIOU
FCN - AlexNet	39.8
FCN - VGG16	56.0
FCN - GoogLeNet	42.5

Existing Implementations

• Keras-FCN Implementation 1
• Keras-FCN Implementation 2

Other Information I think we should have a discussion on the API for the model. Backbones can vary significantly for this model although an almost-standardized method uses the VGG16 and VGG19 models.

@IMvision12 has already begun working on a draft implementation for the VGG16 backbone. We will be collaborating and co-authoring this model.

We can keep the ResNet-50, VGG16 and VGG19 models as supported backbones for the model with their model configs present for use.

Would we want to allow custom backbones? If so, how would we do that?

• Extract convolutional layers/blocks from an existing tf.keras.Model instance passed as a parameter, which would presumably be a plain CNN.
• Custom, list-based parameters such as depth_kernel_sizes, depth_num_filters, and depth_padding_modes which would be then used to generate the backend in an iterative fashion.
• Accept a list of tf.keras.layers.Conv2D / tf.keras.Sequential which would themselves be the backbones’ implementation

@bhack @ianstenbit @LukeWood @tanzhenyu

[tanzhenyu]

Would we want to allow custom backbones? If so, how would we do that?

That’s planned for object detection, and IMO, should be for semantic segmentation too if not already. In #1128, the default backbone is a ResNet101, but the user can supply a keras_cv.models.Model and define the layer names to be used for low-level and high-level feature maps. For example:

backbone = keras_cv.models...

model = segmentation.DeepLabV3Plus(
            classes=11,
            include_rescaling=True,
            backbone=backbone,
            feature_layers=("v2_stack_1_block4_1_relu", "v2_stack_3_block2_2_relu"),
            input_shape=(256, 256, 3),
        )

Alright, makes sense. Are we sure we want to go ahead with this?

We would like all our meta architectures (detection architectures, segmentation architectures, and any future architectures) to take custom backbones

[DavidLandup0]

Would we want to allow custom backbones? If so, how would we do that?

That’s planned for object detection, and IMO, should be for semantic segmentation too if not already. In #1128, the default backbone is a ResNet101, but the user can supply a keras_cv.models.Model and define the layer names to be used for low-level and high-level feature maps.

For example:

backbone = keras_cv.models...

model = segmentation.DeepLabV3Plus(
            classes=11,
            include_rescaling=True,
            backbone=backbone,
            feature_layers=("v2_stack_1_block4_1_relu", "v2_stack_3_block2_2_relu"),
            input_shape=(256, 256, 3),
        )