Misaligned labels saved using NiftiSaver and viewed in 3D Slicer

Describe the bug I used the spleen_segmentation_3d.ipynb tutorial to train a network using a custom dataset (CT scans of hands). When I open the segmentation labels using 3D Slicer, the segments are not aligned with the original scans.

To Reproduce I used the code from the unet_evaluation_dict.py script in the 3D segmentation folder and added it to the evaluation cell of the spleen segmentation Jupyter notebook.

saver = NiftiSaver(output_dir="./output") for val_data in val_loader: … saver.save_batch(val_outputs, val_data["image_meta_dict"])

I then load the original scan as a volume and the corresponding labels (NIfTI file) predicted by the model as a segmentation in 3D Slicer.

Expected behavior See the labels overlap correctly with my scan.

Screenshots In practice, I see a clear translation. Plus, the boundaries of the labels are really strange.

However, if I disable the resampling done by NiftiSaver (resample = False) and load the preprocessed version of the CT scan, everything looks great:

I suspect that the problem comes from the resampling of the labels to match the resolution of the original CT scans. However, when I check the MRML node information in the “Data” module of 3D Slicer, both nodes have the same spacing and the exact same origin.

Environment

================================ Printing MONAI config…

MONAI version: 0.5.0 Numpy version: 1.19.2 Pytorch version: 1.8.1+cu102 MONAI flags: HAS_EXT = False, USE_COMPILED = False MONAI rev id: 2707407fed8c78ccb1c18d1e994a68580457219e

Optional dependencies: Pytorch Ignite version: NOT INSTALLED or UNKNOWN VERSION. Nibabel version: 3.2.1 scikit-image version: 0.17.2 Pillow version: 7.2.0 Tensorboard version: 2.4.1 gdown version: NOT INSTALLED or UNKNOWN VERSION. TorchVision version: NOT INSTALLED or UNKNOWN VERSION. ITK version: NOT INSTALLED or UNKNOWN VERSION. tqdm version: 4.59.0 lmdb version: NOT INSTALLED or UNKNOWN VERSION. psutil version: 5.8.0

For details about installing the optional dependencies, please visit: https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies

================================ Printing system config…

System: Linux Linux version: Ubuntu 20.04.2 LTS Platform: Linux-5.8.0-48-generic-x86_64-with-glibc2.31 Processor: x86_64 Machine: x86_64 Python version: 3.9.2 Process name: python Command: [‘python’, ‘-c’, ‘import monai; monai.config.print_debug_info()’] Open files: [] Num physical CPUs: 8 Num logical CPUs: 16 Num usable CPUs: 16 CPU usage (%): [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 100.0] CPU freq. (MHz): 4799 Load avg. in last 1, 5, 15 mins (%): [0.6, 0.5, 0.1] Disk usage (%): 33.1 Avg. sensor temp. (Celsius): UNKNOWN for given OS Total physical memory (GB): 31.1 Available memory (GB): 24.0 Used memory (GB): 6.5

================================ Printing GPU config…

Num GPUs: 1 Has CUDA: True CUDA version: 10.2 cuDNN enabled: True cuDNN version: 7605 Current device: 0 Library compiled for CUDA architectures: [‘sm_37’, ‘sm_50’, ‘sm_60’, ‘sm_70’] GPU 0 Name: GeForce RTX 2080 SUPER GPU 0 Is integrated: False GPU 0 Is multi GPU board: False GPU 0 Multi processor count: 48 GPU 0 Total memory (GB): 7.8 GPU 0 Cached memory (GB): 0.0 GPU 0 Allocated memory (GB): 0.0 GPU 0 CUDA capability (maj.min): 7.5

Additional context I also tried to set output_dtype = np.short when instantiating my NiftiSaver() (as it’s the type usually used for labels in 3D Slicer), but it didn’t fix the translation issue. It just deteriorated the resampling quality.

[rijobro]

@dianemarquette I’ll create a new issue to check into using batch inversion combined with batch image saving. It’s something I’ve not looked at yet, but obviously would be great if they worked well together!

[dianemarquette]

Thanks for your suggestion @rijobro! It did the trick 😃.

I have a last question if I may. As the tutorial on inverse transforms didn’t show how to save the model predictions, I tried to use the save_batch method from the NiftiSaver class.

model, _, _ = load_ckp(ckp_path, model, optimizer)
model.eval()
with torch.no_grad():
    saver =  NiftiSaver(output_dir = "/home/diane/Documents/Predicted_labels", resample=False, output_dtype = np.short)
    batch_inverter = BatchInverseTransform(val_transforms, val_loader)
    for i, val_data in enumerate(val_loader):
        batch_infer = infer_seg(val_data["image"].to(device), model).detach().cpu()
        segs_dict = {"label": batch_infer,
                     "label_transforms": val_data["label_transforms"],
                     "label_meta_dict": val_data["label_meta_dict"],}
        with allow_missing_keys_mode(val_transforms):
            fwd_bck_batch_labels = batch_inverter(segs_dict)
        saver.save_batch(fwd_bck_batch_labels["label"], fwd_bck_batch_labels["label_meta_dict"])

Unfortunately, it gave me the following error as the batch_labels returned by the batch_inverter are in an iterable.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-21-a9337167fdca> in <module>
     24         # for idx, inverted_seg in enumerate(fwd_bck_batch_labels):
     25             # saver.save(inverted_seg["label"], inverted_seg["label_meta_dict"])
---> 26         saver.save_batch(fwd_bck_batch_labels["label"], fwd_bck_batch_labels["label_meta_dict"])

TypeError: list indices must be integers or slices, not str

Replacing the last line by:

for idx, inverted_seg in enumerate(fwd_bck_batch_labels):
            saver.save(inverted_seg["label"], inverted_seg["label_meta_dict"])

made it work, but I was wondering if there was a way to implement the inverted predictions storage by batch using save_batch.

Thanks !