How to change VGG16.py to wire conv4_3 instead of conv5_3?

I have trained faster rcnn with a custom dataset based on Pascal VOC format. Now I wanted to use 3rd or 4th convolutional layers of vgg16 to deal with the object of specific size. But I don’t know how to change the vgg16 net exactly. I tried to remove the final lines of this section:

  def _image_to_head(self, is_training, reuse=False):
    with tf.variable_scope(self._scope, self._scope, reuse=reuse):
      net = slim.repeat(self._image, 2, slim.conv2d, 64, [3, 3],
                          trainable=False, scope='conv1')
      net = slim.max_pool2d(net, [2, 2], padding='SAME', scope='pool1')
      net = slim.repeat(net, 2, slim.conv2d, 128, [3, 3],
                        trainable=False, scope='conv2')
      net = slim.max_pool2d(net, [2, 2], padding='SAME', scope='pool2')
      net = slim.repeat(net, 3, slim.conv2d, 256, [3, 3],
                        trainable=is_training, scope='conv3')
      net = slim.max_pool2d(net, [2, 2], padding='SAME', scope='pool3')
      net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3],
                        trainable=is_training, scope='conv4')
      net = slim.max_pool2d(net, [2, 2], padding='SAME', scope='pool4')
      net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3],
trainable=is_training, scope='conv5')

but it didn’t work. Where else should be modified to make it connect earlier layers of vgg16 to the RPN?

I would appreciate some help from experts on this. thanks

[yaoqi-zd]

@zqdeepbluesky you can check the feature fusion methods used in HyperNet or FPN(feature pyramid network)

[yaoqi-zd]

@hadi-ghnd em…I thought although the lower feature map(like conv4_3) has higher resolution, they have weaker semantic info, which may harm the localization performance. Have you tried to add a deconv layer to the conv5_3 to make conv5_3 the same size of conv4_3(let’s say this feature map as deconv5_3) and then combine conv4_3 and deconv5_3 to get a featrue map that has both higher resolution and strong semantics?