Visualizing heatmaps of class activation

Based on this deep-learning-with-python-notebooks implementation of CAM I have the following issue:

I have fine-tuned ResNet50 on my 9 new categories and have saved my model in .h5 format.

Then I am loading the saved model and find out the correct entry in the prediction vector without problems. My code then is the following:

# The is the output feature map of the `block5_conv3` layer,
# the last convolutional layer in ResNet50
last_conv_layer = model.get_layer('conv2d_14')

# This is the gradient of the "my_label" class with regard to
# the output feature map of `block5_conv3`
grads = K.gradients(my_label_output, last_conv_layer.output)[0]

where my_label_output = model.output[:, 1]

after that

# This is a vector of shape (60,), where each entry
# is the mean intensity of the gradient over a specific feature map channel
pooled_grads = K.mean(grads, axis=(0, 1, 2))

print initial_model.input.shape
print pooled_grads.shape
print last_conv_layer.output[0].shape

# This function allows us to access the values of the quantities we just defined:
# `pooled_grads` and the output feature map of `conv2d_14`,
# given a sample image
iterate = K.function([initial_model.input], [pooled_grads, last_conv_layer.output[0]])

# These are the values of these two quantities, as Numpy arrays,
# given our sample image of two elephants
pooled_grads_value, conv_layer_output_value = iterate([x])

but then I get the following error:

Invalid argument: Shape [-1,7,7,2048] has negative dimensions

Here is the summary of my loaded model

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_7 (InputLayer)         (None, 7, 7, 2048)        0         
_________________________________________________________________
conv2d_13 (Conv2D)           (None, 7, 7, 30)          61470     
_________________________________________________________________
conv2d_14 (Conv2D)           (None, 7, 7, 60)          1860      
_________________________________________________________________
average_pooling2d_7 (Average (None, 1, 1, 60)          0         
_________________________________________________________________
flatten_7 (Flatten)          (None, 60)                0         
_________________________________________________________________
dense_7 (Dense)              (None, 9)                 549       
=================================================================
Total params: 63,879
Trainable params: 63,879
Non-trainable params: 0

Any thoughts on this ?

[GKalliatakis]

@vianamp Can you give me an example of a fine-tuned model ?

The code is working fine when I use the models from keras applications, but I want to use it on one of my fine-tuned models.

Many thanks

[JosephPai]

Hi @GKalliatakis I have fine-tuned ResNet50 on a 6 categories task and want a heap-map from it, then got a similar problem. Could you please paste your codes about building and training of the model, it’s better if there is the codes of getting heat-map. Many thanks !!!