question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

How to merge two DirectoryIterator?

See original GitHub issue

Hi, I’m looking at the example here: https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html

They create 3 iterators: for train, validation and test.

test_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
        'data/train',  # this is the target directory
        target_size=(150, 150),  # all images will be resized to 150x150
        batch_size=batch_size,
        class_mode='binary')  # since we use binary_crossentropy loss, we need binary labels

validation_generator = test_datagen.flow_from_directory(
        'data/validation',
        target_size=(150, 150),
        batch_size=batch_size,
        class_mode='binary')

My question is whether it’s possible to merge\combine train_generator with validation_generator to a single generator that contains images from both train and validation folder.

THX

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:7 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
Dref360commented, Sep 11, 2018

Iterators are Sequence. You can create you own Sequence

from keras.utils import Sequence

flow1 = train_datagen.flow_from_directory(...)
flow2 = train_datagen.flow_from_directory(...)

class MySequence(Sequence):
    def __init__(self, seq1, seq2):
        self.seq1, self.seq2 = seq1, seq2
    def __len__(self):
        return len(self.seq1)

    def __getitem(self, idx):
        x1, y1 = self.seq1[idx]
        x2, y2 = self.seq2[idx]
        return [x1, x2], [y1, y2]

my_seq = MySequence(flow1, flow2)
model.fit_generator(my_seq, ...)
0reactions
vipuljain-17commented, Jun 17, 2020

Did you ever get the solution to this problem?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Joining two DirectoryIterators in Keras - Stack Overflow
Just combine the generators in another generator, optionally with different augmentation configs: idg1 = ImageDataGenerator(**idg1_configs) ...
Read more >
tf.keras.preprocessing.image.DirectoryIterator - TensorFlow
Iterator capable of reading images from a directory on disk.
Read more >
Feature #11: Directory Iterator - Decode the Coding Interview ...
Implementing the "Directory Iterator" feature for our "Operating System" project. ... Feature #2: Merge Tweets In Twitter Feed.
Read more >
DirectoryIterator::isFile - Manual - PHP
DirectoryIterator ::isFile — Determine if current DirectoryIterator item is a regular file ... Merge the flipped arrays to get rid of duplicate
Read more >
Image Preprocessing - Keras Documentation - API Manual
brightness_range: Tuple or list of two floats. ... class_mode=None, seed=seed) # combine generators into one which yields image and masks train_generator ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found