Quantization Not Fusing Pad to Conv2D

Recently, I quantized a pre-trained ResNet50 model from fp32 to int8, and I noticed that the performance isn’t what I expected. The performance is only about 2x compared to the equivalent fp32 model. Investigating further, I noticed in Netron that INC didn’t seem to fuse Pad with any later operations.

I’ve got two int8 models:

1. Model A: Model that I downloaded from the Intel Model Zoo (https://github.com/IntelAI/models/blob/master/benchmarks/image_recognition/tensorflow/resnet50/inference/int8/README.md).
2. Model B: Model that I quantized using INC, using the example instructions (https://github.com/intel/neural-compressor/tree/master/examples/tensorflow/object_detection/tensorflow_models/quantization/ptq).

Model A is available from the above link, but here’s Model B. GitHub wouldn’t allow me to upload PB files, so I changed the file extension to .zip.

My question is this: does INC support fusing Pad operations, or am I running the scripts incorrectly?

[ftian1]

thanks for raising such an issue.

INC supports Pad + conv fusion. The problem is if the pad op’s tensor value is not all 0, it may trigger TensorFlow int8 op shape inference check and break. The fix on TF kernel was developed by Intel TensorFlow team and only merged in 1.15up3 but not TF 2.x. If you try to use Intel TensorFlow 1.15 up3 version, you can see the pb having fused ops.

pls note INC is for broad model but not specific optimization

[lvliang-intel]

@matthew-olson-intel The PR https://github.com/tensorflow/tensorflow/pull/53480 is related with Pad + Conv3D. It’s not for Pad + Conv2D. Good news is that this issue has already been fixed by TF SPR-Base and will upstream to stock TF soon. You can try the quantization with SPR-Base as aligned in Teams. Please let’s know if you have any issue.