Cannot employ a Lambda transform into SubjectsDatasets for Queue
See original GitHub issue🐛Bug
Hi guys, thanks for the library is quite cool!
I want to employ torchio in a 2D architecture so I am employing the Queue dataset in order to extract patches that represent one slice. Everything works fine when I do not apply transforms or I employ your built-in transforms. However, since I am using a pre-pre-trained model with a different slice orientation I am trying to apply an easy Lambda which just employs torch.transpose. In this case, I get the error showed at traces paragraph.
Just in case let me add that the Lambda transform alone works OK
To reproduce
transpose_transform = tio.Lambda(lambda x: torch.transpose(x, 1,2))
transformadas = tio.Compose([transpose_transform]) #Without or with built-in transforms works properly
dataset_transform = tio.SubjectsDataset(subjects, transform=transformadas) # I tried with just the transpose (instead a composition) transform but I got the same result
slices_queue = tio.Queue(
subjects_dataset=dataset_transform,
max_length=24,
samples_per_volume=4, # For this test I employ a datsset just with 6 3D-images
sampler=tio.sampler.UniformSampler((500,400,1)),
num_workers=6,
shuffle_subjects=False,
shuffle_patches=True)
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function <lambda> at 0x7f3d89c6fb70>: attribute lookup <lambda> on __main__ failed
File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function <lambda> at 0x7f3d89c6fb70>: attribute lookup <lambda> on __main__ failed
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function <lambda> at 0x7f3d89c6fb70>: attribute lookup <lambda> on __main__ failed
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function <lambda> at 0x7f3d89c6fb70>: attribute lookup <lambda> on __main__ failed
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function <lambda> at 0x7f3d89c6fb70>: attribute lookup <lambda> on __main__ failed
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function <lambda> at 0x7f3d89c6fb70>: attribute lookup <lambda> on __main__ failed
Expected behavior
Work as with the built-in transforms
“Can’t pickle”
TorchIO version 0.18.0
Issue Analytics
- State:
- Created 3 years ago
- Comments:10 (6 by maintainers)
Top Results From Across the Web
Troubleshooting Lambda configurations - AWS Documentation
Memory configurations. You can configure a Lambda function to use between 128 MB and 10,240 MB. By default, any function created in the...
Read more >Processing Large S3 Files With AWS Lambda - Medium
Lambda is a good option if you want a serverless architecture and have files that are large but still within reasonable limits. It...
Read more >Python multiprocessing PicklingError: Can't pickle <type ...
The problem is that the pool methods all use a mp.SimpleQueue to pass tasks to the worker processes. Everything that goes through the...
Read more >3 Pitfalls of AWS Step Functions and How You Can Avoid Them
When starting out with Step Functions, it is entirely possible to use Lambda functions, SQS queues, SNS topics, etc… without regard to which ......
Read more >Data — MONAI 1.1.0 Documentation
Extend the IterableDataset with a buffer and randomly pop items. Parameters. data – input data source to load and transform to generate dataset...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Yeah!
Hi, @Geeks-Sid. Could you please open a new issue and share a minimal, reproducible example that I can run on my Linux computer? That will help me solve your issue.