Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Image/label values normalisation

See original GitHub issue

Issue description

Sometimes, the data from nifti or from h5 might not have the range as we expected.

For now, we expect:

  • Images have values between 0-255
  • Labels have values between 0-1

And so we will divide the image tensors by 255.

But sometimes this might not be the case and maybe user would like to define their own value range. So we can do the following instead,

  • For each file loader, h5 or nifti, they will need a value range [min, max] and they will normalize the arrays using the given values by x = (x - min) / (max - min + EPS) with EPS=1.0E-6 a very small number. The min and max can be None (any one of them, not necessarily both), and if None, the min/max will be calculated dynamically on the fly. So the min/max would not be global, but be array dependant.
  • Then, for image file loader we will assume the max = 255, min = 0, for label file loader we will assume the max = 1, min = 0. However, user can overwrite this values via config. But we do not have to put these values in config, it’s optional. Otherwise the config will be too complicated.

What do you think? @ebonmati @NMontanaBrown @YipengHu

Type of Issue

  • New feature request
  • Documentation update
  • Test request

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

YipengHucommented, Jul 25, 2020

I divide it by 255 because I don’t want to feed it with different ranges. So fine, agree that with batch norm the difference is little. Will change so, and change the configs to make it clear, unless given min/max in the config, we will normalize dynamically.

@mathpluscode i see your point - just conventions - pick one and clarify it in config/doc.

YipengHucommented, Jul 24, 2020

It is error-prone with anything “we assume”. i think by default, it should be x = (x - min) / (max - min + EPS) - calculated per image, unless a global customised [min max] is given; - no need for 255 as a special case.

Another important normalisation is unit-variance with zero-mean. Not sure i see clear advantage, so let’s leave it post-r1 when there is a need.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Keras MobileNetv2 image normalization · Issue #209
Hi, I train my model using keras, and I load data using this function Resize the images to a fixed input size, and...
Read more >
Normalizing BatchDataset in Tensorflow 2.3 - python
Try this way: def process(image,label): image = tf.cast(image/255. ,tf.float32) return image,label ds ...
Read more >
Assessing the Impact of Color Normalization in ...
Therefore, post-normalization a positive CD value would indicate an increase ... Given the nuclei image label, L, and nuclei prediction, P, ...
Read more >
How to Normalize, Center, and Standardize Image Pixels ...
How to use the ImageDataGenerator to normalize pixel values when fitting and evaluating a convolutional neural network model.
Read more >
Normalize Image — v5.3.0
Normalize an image by setting its mean to zero and variance to one. NormalizeImageFilter shifts and scales an image so that the pixels...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found