Keras TF ImageNet Preprocessing contradicts TF, may affect pretrained weights

It appears Keras’ imagenet image preprocessing may be inconsistent with how it is done in tf, in particular keras sets values to [-1, 1] while in tf the expected range is [0, 1].

• slim/preprocessing/inception_preprocessing.py
  • notes If dtype is tf.float32 then the range should be [0, 1]
• keras preprocess_input()
  • notes tf: will scale pixels between -1 and 1, sample-wise.

Here is the key doc from Keras’ preprocess_input():

def preprocess_input(x, data_format=None, mode='caffe'):
    """Preprocesses a tensor encoding a batch of images.
    # Arguments
        x: input Numpy or symoblic tensor, 3D or 4D.
        data_format: data format of the image tensor.
        mode: One of "caffe", "tf".
            - caffe: will convert the images from RGB to BGR,
                then will zero-center each color channel with
                respect to the ImageNet dataset,
                without scaling.
            - tf: will scale pixels between -1 and 1,
                sample-wise.
    # Returns
        Preprocessed tensor.
    """

Here are the key docs from the tf slim function preprocess_for_train() which specify a [0, 1] range:

def preprocess_for_train(image, height, width, bbox,
                         fast_mode=True,
                         scope=None,
                         add_image_summaries=True):
  """Distort one image for training a network.
  Distorting images provides a useful technique for augmenting the data
  set during training in order to make the network invariant to aspects
  of the image that do not effect the label.
  Additionally it would create image_summaries to display the different
  transformations applied to the image.
  Args:
    image: 3-D Tensor of image. If dtype is tf.float32 then the range should be
      [0, 1], otherwise it would converted to tf.float32 assuming that the range
      is [0, MAX], where MAX is largest positive representable number for
      int(8/16/32) data type (see `tf.image.convert_image_dtype` for details).
    height: integer
    width: integer
    bbox: 3-D float Tensor of bounding boxes arranged [1, num_boxes, coords]
      where each coordinate is [0, 1) and the coordinates are arranged
      as [ymin, xmin, ymax, xmax].
    fast_mode: Optional boolean, if True avoids slower transformations (i.e.
      bi-cubic resizing, random_hue or random_contrast).
    scope: Optional scope for name_scope.
    add_image_summaries: Enable image summaries.
  Returns:
    3-D float Tensor of distorted image used for training with range [-1, 1].
  """

side notes:

• https://github.com/tensorflow/tensorflow/issues/15722 is the corresponding tf issue
• the tf image adjustments guide doesn’t have much documenting the inputs.

[ahundt]

@ztf-ucas I ran lots of automated experiments with both to try and find an optimal choice, and I recall it honestly didn’t make a big difference in my use case. I simply default to ‘tf’ now, but as with all neural networks your mileage may vary based on your dataset and problem type.

[ymodak]

Yes you are right. Thanks for catching this. Reopening the issue since its still remains.