The network size is not reduced(same FP32 version)

Describe the bug

Hi all,

I’m doing my research with this repo. I have made Binarized version of ResNet18 and trained it with CIFAR10 dataset whose size is (224,224,3) and got the final top-1 accuracy is 91%.

after training, I converted the model to Tensorflow Lite model, but there was no size reduction after converting at all.

But when I tried to convert the example model which is described in here, Its size was reduced what I expected(39.52MB -> 1.28MB).

I want to know what is wrong with my model.

my model is defined following code:

from multiprocessing.sharedctypes import Value
import tensorflow as tf
from tensorflow.keras.layers import Input, Conv2D, BatchNormalization, Dense, MaxPool2D, GlobalAveragePooling2D
import larq

tf.random.set_seed(777)
class BResNet(tf.keras.Model):
    '''
    Binary ResNet
    Author: H.J. Shin
    Date: 2022.04.05
    '''
    def __init__(self, classes=10, arch='S'):
        '''
        classes: number of logits, default=1000(Imagenet)
        '''
        super(BResNet, self).__init__()
        
        self.conv1 = Conv2D(filters=64, kernel_size=7, strides=2, padding='same', use_bias=False)
        self.maxpool1 = MaxPool2D(pool_size=1, strides=2)
        
        self.conv2_1 = BResblock(num_channels=64, use_1x1conv=False, strides=1)
        self.conv2_2 = BResblock(num_channels=64, use_1x1conv=False, strides=1)
        
        self.conv3_1 = BResblock(num_channels=128, use_1x1conv=True,strides=2)
        self.conv3_2 = BResblock(num_channels=128, use_1x1conv=False,strides=1)
        
        self.conv4_1 = BResblock(num_channels=256, use_1x1conv=True, strides=2)
        self.conv4_2 = BResblock(num_channels=256, use_1x1conv=False, strides=1)
        
        self.conv5_1 = BResblock(num_channels=512,use_1x1conv=True, strides=2)
        self.conv5_2 = BResblock(num_channels=512,use_1x1conv=False,strides=1)
        
        self.global_avg_pool = GlobalAveragePooling2D()
        self.linear = Dense(classes, use_bias=False, activation='softmax')

        self.relu = tf.nn.relu
        
    def call(self, x):
        
        y = self.conv1(x)
        y = self.maxpool1(y)
        y = self.relu(y)

        y = self.conv2_1(y)
        y = self.conv2_2(y)

        y = self.conv3_1(y)
        y = self.conv3_2(y)
        
        y = self.conv4_1(y)
        y = self.conv4_2(y)
        
        y = self.conv5_1(y)
        y = self.conv5_2(y)
        
        y = self.global_avg_pool(y)
        
        logtis = self.linear(y)      
       
        return logtis
    
    def model(self, input_shape):
        '''
        This method makes the command "model.summary()" work.
        input_shape: (H,W,C), do not specify batch B
        '''
        x = Input(shape=input_shape)
        model = tf.keras.Model(inputs=[x], outputs=self.call(x))
        print(larq.models.summary(model))
        return model

class BResblock(tf.keras.layers.Layer):
    '''
    Residual Block
    
    Author: H.J. Shin
    Date: 2022.04.05
    '''
    def __init__(self, num_channels, use_1x1conv=False, strides=1):
        super(BResblock, self).__init__()
        
        self.num_channels= num_channels
        self.use_1x1conv = use_1x1conv
        self.strides = strides
        
    def build(self, input_shape):

        B,H,W,C = input_shape
        
        
        self.conv1 = larq.layers.QuantConv2D(filters=self.num_channels, kernel_size=3, padding='same', strides=self.strides, use_bias=False,
                                      kernel_quantizer="ste_sign",
                                      kernel_constraint="weight_clip",)
                                      #input_quantizer="ste_sign")
        self.conv2 = larq.layers.QuantConv2D(filters=self.num_channels, kernel_size=3, padding='same', strides=1, use_bias=False,
                                      kernel_quantizer="ste_sign",
                                      kernel_constraint="weight_clip",)
                                      #input_quantizer="ste_sign")
        self.conv3 = None
        if self.use_1x1conv:
            self.conv3 = larq.layers.QuantConv2D(self.num_channels, kernel_size=1, strides=self.strides,
                                kernel_quantizer="ste_sign",
                                kernel_constraint="weight_clip",)
                                #input_quantizer="ste_sign")
        
        self.bn1 = BatchNormalization()
        self.bn2 = BatchNormalization()
    def call(self, x):
        
        y = tf.nn.relu(self.bn1(self.conv1(x)))
        y = self.bn2(self.conv2(y))
        if self.conv3 is not None:
            x = self.conv3(x)
        
        y += x
    
        return tf.nn.relu(y) 

And larq.models.summary(model) says as:

+model stats------------------------------------------------------------------------------------------+
| Layer                     Input prec.             Outputs   # 1-bit  # 32-bit   Memory  32-bit MACs |
|                                 (bit)                           x 1       x 1     (kB)              |
+-----------------------------------------------------------------------------------------------------+
| input_1                             -   (-1, 224, 224, 3)         0         0        0            ? |
| conv2d                              -  (-1, 112, 112, 64)         0      9408    36.75    118013952 |
| max_pooling2d                       -    (-1, 56, 56, 64)         0         0        0            0 |
| tf.nn.relu                          -    (-1, 56, 56, 64)         0         0        0            ? |
| b_resblock                          -    (-1, 56, 56, 64)     73728       512    11.00            ? |
| b_resblock_1                        -    (-1, 56, 56, 64)     73728       512    11.00            ? |
| b_resblock_2                        -   (-1, 28, 28, 128)    229376      1152    32.50            ? |
| b_resblock_3                        -   (-1, 28, 28, 128)    294912      1024    40.00            ? |
| b_resblock_4                        -   (-1, 14, 14, 256)    917504      2304   121.00            ? |
| b_resblock_5                        -   (-1, 14, 14, 256)   1179648      2048   152.00            ? |
| b_resblock_6                        -     (-1, 7, 7, 512)   3670016      4608   466.00            ? |
| b_resblock_7                        -     (-1, 7, 7, 512)   4718592      4096   592.00            ? |
| global_average_pooling2d            -           (-1, 512)         0         0        0            ? |
| dense                               -            (-1, 10)         0      5120    20.00         5120 |
+-----------------------------------------------------------------------------------------------------+
| Total                                                      11157504     30784  1482.25    118019072 |
+-----------------------------------------------------------------------------------------------------+
+model summary-----------------------------+
| Total params                   11.2 M    |
| Trainable params               11.2 M    |
| Non-trainable params           7.68 k    |
| Model size                     1.45 MiB  |
| Model size (8-bit FP weights)  1.36 MiB  |
| Float-32 Equivalent            42.68 MiB |
| Compression Ratio of Memory    0.03      |
| Number of MACs                 118 M     |
+------------------------------------------+

To Reproduce

To convert from Tensorflow model to Tensorflow Lite model, I used following python codes:

model = tf.keras.models.load_model('./models/BResNet_CIFAR10_224_2022-04-05_17-01-46/')

with lq.context.quantized_scope(True):
    model.save('test_bnn')

with open('test_bnn.tflite', 'wb') as f:
    tflite = lce.convert_saved_model('test_bnn')
    f.write(tflite)

Expected behavior

I hope that my model has size 1.45 MiB which is described in larq.models.summary() method.

Environment

TensorFlow version: 2.8.0 Larq version: 0.12.2

Many Thanks.

[Tombana]

Your understanding is correct. Our tflite operators do not support binary-weight convolutions, only fully-binary convolutions. That’s why the converter will leave them as float, so that they will still run correctly.

[Tombana]

That is correct. XNOR and popcount only works if both the inputs and weights are binary.