question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

'Dataloder' object is not an iterator

See original GitHub issue

Can someone help why I am getting this error? I am using a PSPNet model with 2 classes for my custom dataset. Here is the stack trace:

`

TypeError Traceback (most recent call last) <ipython-input-21-9bb7e6991b88> in <module> 1 print(len(train_dataloader)) ----> 2 fitted_model= model.fit_generator(train_dataloader, steps_per_epoch=len(train_dataloader), epochs=10, callbacks=callbacks,)

~/anaconda3/lib/python3.7/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs) 89 warnings.warn('Update your ' + object_name + ' call to the ’ + 90 'Keras 2 API: ’ + signature, stacklevel=2) —> 91 return func(*args, **kwargs) 92 wrapper._original_function = func 93 return wrapper

~/anaconda3/lib/python3.7/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch) 1416 use_multiprocessing=use_multiprocessing, 1417 shuffle=shuffle, -> 1418 initial_epoch=initial_epoch) 1419 1420 @interfaces.legacy_generator_methods_support

~/anaconda3/lib/python3.7/site-packages/keras/engine/training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch) 179 batch_index = 0 180 while steps_done < steps_per_epoch: –> 181 generator_output = next(output_generator) 182 183 if not hasattr(generator_output, ‘len’):

~/anaconda3/lib/python3.7/site-packages/keras/utils/data_utils.py in get(self) 707 “use_multiprocessing=False, workers > 1.” 708 “For more information see issue #1638.”) –> 709 six.reraise(*sys.exc_info())

~/anaconda3/lib/python3.7/site-packages/six.py in reraise(tp, value, tb) 691 if value.traceback is not tb: 692 raise value.with_traceback(tb) –> 693 raise value 694 finally: 695 value = None

~/anaconda3/lib/python3.7/site-packages/keras/utils/data_utils.py in get(self) 683 try: 684 while self.is_running(): –> 685 inputs = self.queue.get(block=True).get() 686 self.queue.task_done() 687 if inputs is not None:

~/anaconda3/lib/python3.7/multiprocessing/pool.py in get(self, timeout) 655 return self._value 656 else: –> 657 raise self._value 658 659 def _set(self, i, obj):

~/anaconda3/lib/python3.7/multiprocessing/pool.py in worker(inqueue, outqueue, initializer, initargs, maxtasks, wrap_exception) 119 job, i, func, args, kwds = task 120 try: –> 121 result = (True, func(*args, **kwds)) 122 except Exception as e: 123 if wrap_exception and func is not _helper_reraises_exception:

~/anaconda3/lib/python3.7/site-packages/keras/utils/data_utils.py in next_sample(uid) 624 The next value of generator uid. 625 “”" –> 626 return six.next(_SHARED_SEQUENCES[uid]) 627 628

TypeError: ‘Dataloder’ object is not an iterator `

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:9

github_iconTop GitHub Comments

1reaction
sainatarajancommented, Sep 19, 2019

@JordanMakesMaps Thank you !!! I have managed to run and get a decent output (fewer data and small epochs). As you said I solved the shape error by editing the model definition by changing the number of classes to 3. This lead to another error: Error: Attempting to use uninitialized value of Adam_Lr21. I had solved it by removing the callbacks from the fit_generator() method. I am closing this issue now.

0reactions
JordanMakesMapscommented, Sep 19, 2019

train and valid are just pandas’ dataframes with two columns: ‘Images’ and ‘Masks’ which are paths to the files themselves.

ValueError: Error when checking target: expected softmax to have shape (768, 1152, 2) but got array with shape (768, 1152, 3)

That might be caused by how you provide the training mask to the model or, how many classes the model expects when first initiated. In my Generator class, during the preprocessing phase I create a one-hot-encoded vector to represent the mask in the form that Keras expects. Are you using to_categorical(), the function provided by Keras? Either way, you might want to look at what the output shape is of the one-hot-encoded vector as it might be the source of the problem.

Read more comments on GitHub >

github_iconTop Results From Across the Web

DataLoader iterator is not working for Custom Dataset
I am creating a custom dataset and data loader. Unfortunately, when I apply next(iter(dataloader), it gives the following error.
Read more >
Iterator doesn't work with DataLoader on GPU - Stack Overflow
For google –I got here after getting > AttributeError: '_SingleProcessDataLoaderIter' object has no attribute 'next'.
Read more >
Iterating through DataLoader using iter() and next() in PyTorch
Iterator object allows you to traverse through all the elements of a DataLoader, regardless of its specific implementation. An iterator is an ...
Read more >
pytorch从dataloader中取数据(python从enumerate或iterator ...
报错:TypeError: 'enumerate' object is not subscriptableTypeError: 'DataLoader' object is not an iterator代码:_, (inputs2, ...
Read more >
tf.data.Iterator | TensorFlow v2.11.0
Represents an iterator of a tf.data.Dataset. ... Optional will have no value. ... Optional object representing the next element.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found