question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

AugmentationSequential does not return lists of boxes

See original GitHub issue

Describe the bug

When passing boxes as lists of tensors with any BBOX datakey to AugmentationSequential the resulting augmentation is not returned. Thus, the returned outputs have one fewer element than the inputs that were passed to it. This happens for both forward and inverse.

Reproduction steps

import kornia.augmentation as K
import torch

img = torch.rand((3,3,10,10))
bbox = [torch.tensor([[1,5,2,7],[0,3,9,9]]),torch.tensor([[1,5,2,7],[0,3,9,9],[0,5,8,7]]), torch.empty((0,4))]
inputs = [img, bbox]

aug = K.AugmentationSequential(K.Resize((300,300)), data_keys=['input', 'bbox_xyxy'])

transformed = aug(*inputs)

assert len(transformed) == len(inputs)
Traceback (most recent call last):             
  File "/home/miquelmr/kornia/test.py", line 12, in <module>
    assert len(transformed) == len(inputs)
AssertionError  

Expected behavior

List of boxes is returned.

Environment

Collecting environment information...
PyTorch version: 1.13.0+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.27

Python version: 3.10.8 (main, Nov 24 2022, 14:13:03) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-4.15.0-196-generic-x86_64-with-glibc2.27
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: NVIDIA TITAN X (Pascal)
GPU 1: NVIDIA TITAN X (Pascal)
GPU 2: NVIDIA TITAN Xp

Nvidia driver version: 520.61.05
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.23.5
[pip3] torch==1.13.0
[pip3] torchvision==0.14.0
[conda] numpy                     1.23.5                   pypi_0    pypi
[conda] torch                     1.13.0                   pypi_0    pypi
[conda] torchvision               0.14.0                   pypi_0    pypi

Additional context

No response

Issue Analytics

  • State:open
  • Created 10 months ago
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
shijianjiancommented, Dec 6, 2022

It seems that the Boxes wrapper works just fine when passing lists of tensors up to when it is converted back to a tensor or a list of tensors here, depending on what the input was:

https://github.com/kornia/kornia/blob/370099180c97167d066d94491eda46e7b9c8b939/kornia/augmentation/container/augment.py#L275

This seems to work fine too but now outputs contains a list of tensors (if that was the format passed to the input), which gets removed from the list of outputs in L279. It is this last removal that I don’t understand the purpose of and the implications that it would have to just not do that, as it would seem to be required to support returning lists of tensors.

I agree here. I think we shall not use any obvious wrapper at all. Say, the user input a list of tensors directly and get the corresponding list of tensors. If they input a wrapped Box object, they should get the corresponding Box object.

Additionally, we should use our own Box data type as the default communication type to pass around different methods.

0reactions
miquelmarticommented, Dec 6, 2022

It seems that the Boxes wrapper works just fine when passing lists of tensors up to when it is converted back to a tensor or a list of tensors here, depending on what the input was: https://github.com/kornia/kornia/blob/370099180c97167d066d94491eda46e7b9c8b939/kornia/augmentation/container/augment.py#L275

This seems to work fine too but now outputs contains a list of tensors (if that was the format passed to the input), which gets removed from the list of outputs in L279. It is this last removal that I don’t understand the purpose of and the implications that it would have to just not do that, as it would seem to be required to support returning lists of tensors.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Augmentation Containers - Kornia
Dispatches different augmentations to a single input and returns a list. Same datakeys must be applied across different augmentations. By default, with input...
Read more >
kornia/augment.py at master - GitHub
This example demonstrates the integration of VideoSequential and AugmentationSequential. >>> import kornia. >>> input = torch.randn(2 ...
Read more >
Data Augmentation For Bounding Boxes: Flipping
It takes two arguments, the image img and the bounding box annotations bboxes and returns the transformed values. This is it for this...
Read more >
Applying the same augmentation with the same parameters to ...
Later, you will use those names to pass additional targets to a transformation pipeline. So you can't use a string that starts with...
Read more >
data augmentation layers makes logits and labels mismatch ...
Any Keras layer will have to keep the batch size the same as the input, so it's not possible as a Keras layer....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found