Call new layer on the last layer of create_model object using Functional API

Hi. First, I want to say that I enjoy this library a lot! Thank you @martinsbruveris for creating it!

I have a question: I want create a model body using create_model function and add my own classification head. In classification head I want to add another input layer to additional features, call a concatenate layer on last layer of the create_model object and new input layer, and add final dense layer. Since create_model object is not a Sequential or Functional model object, is there any way I can do that? I tried using ‘model_tfimm.output’ or ‘model_tfimm.layers[-1].output’ calls, because .output call works with Tensorflow models, but it does not seem to work with tfimm models:

dense_1 = tf.keras.layers.Dense(512, activation='relu', name='dense_1')(model_tfimm.layers[-1].output)

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
v:\Git\spellbook\magicClassification.py in <module>
----> 1 dense_1 = tf.keras.layers.Dense(512, activation='relu', name='dense_1')(model_tfimm.layers[-1].output)

~\AppData\Local\Programs\Python\Python37\lib\site-packages\keras\engine\base_layer.py in output(self)
  2094     """
  2095     if not self._inbound_nodes:
-> 2096       raise AttributeError('Layer ' + self.name + ' has no inbound nodes.')
  2097     return self._get_node_attribute_at_index(0, 'output_tensors', 'output')
  2098 

AttributeError: Layer activation_72 has no inbound nodes.

model_tfimm.output

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
v:\Git\spellbook\magicClassification.py in <module>
----> 1 model_tfimm.output

~\AppData\Local\Programs\Python\Python37\lib\site-packages\keras\engine\base_layer.py in output(self)
   2094     """
   2095     if not self._inbound_nodes:
-> 2096       raise AttributeError('Layer ' + self.name + ' has no inbound nodes.')
   2097     return self._get_node_attribute_at_index(0, 'output_tensors', 'output')
   2098 

AttributeError: Layer conv_ne_xt_1 has no inbound nodes.

Using Tensorflow Functional API this would like something like this:

model_tfimm = tfimm.create_model(TFIMM_MODEL_NAME, nb_classes=0, pretrained="timm")
feature_extractor = model_tfimm.output

add_input = tf.keras.layers.Input(shape=(NUM_ADD_FEATURES, ), name='input_features_layer')
concat_layer = tf.keras.layers.Concatenate(name='concat_features')([feature_extractor, add_input])

predictions = tf.keras.layers.Dense(NUM_CLASSES, activation=OUTPUT_ACTIVATION)(concat_layer)

model = tf.keras.Model(inputs=[model_tfimm.input, add_input], outputs=predictions)

Any ideas?

[martinsbruveris]

Yes, you should be able to train all layers by default. Consider this code

import tensorflow as tf
import tfimm
from tfimm.utils.flops import get_parameters

x = tf.keras.Input((32, 32, 3))
y = tf.keras.Input((512,))
backbone = tfimm.create_model("resnet18", nb_classes=0)
z = backbone(x)
z = tf.keras.layers.Concatenate()([y, z])
z = tf.keras.layers.Dense(units=10)(z)
model = tf.keras.Model(inputs=[x, y], outputs=z)

print(get_parameters(backbone))
print(get_parameters(model))

Note that the combined model has all the trainable parameters of backbone plus the ones from the dense layer.

[martinsbruveris]

Happy to help. I will close the issue for now. Feel free to reopen it, if needed.