question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Add image vision datasets as Base lightningModules

See original GitHub issue

Add the torchvision datasets including the standard transforms and splits as modules


class MNISTModule(pl.LightningModule):

    def prepare_data(self):
        self.mnist_train = MNIST(os.getcwd(), train=True, download=True, transform=transforms.ToTensor())
        self.mnist_test = MNIST(os.getcwd(), train=False, download=True, transform=transforms.ToTensor())

    def train_dataloader(self):
        loader = DataLoader(self.mnist_train, batch_size=self.hparams.batch_size, num_workers=self.hparams.num_workers)
        return loader

    def val_dataloader(self):
        loader = DataLoader(self.mnist_test, batch_size=self.hparams.batch_size, num_workers=self.hparams.num_workers)
        return loader

    def test_dataloader(self):
        loader = DataLoader(self.mnist_test, batch_size=self.hparams.batch_size, num_workers=self.hparams.num_workers)
        return loader

from pytorch_lightning.bolts.dataset_modules import MNISTModule

class MyModel(MNISTModule, pl.LightningModule):

    # all the dataloaders have been implemented with MNIST

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:1
  • Comments:10 (7 by maintainers)

github_iconTop GitHub Comments

2reactions
edgarribacommented, Apr 21, 2020

I agree with @williamFalcon to only include curated datasets and potentially all from torchvision since it has a quite robust API. In case of missing ones, I think it’s best to invest in contributing to torchvision instead of start creating custom stuff. For downloading, I would give the control to the user. Not all of the datasets in torchvision have really this option since in some of them you have to download the data in an offline process.

0reactions
williamFalconcommented, Jun 28, 2020

we need to evaluate direct support with PL after we test this for a bit

Read more comments on GitHub >

github_iconTop Results From Across the Web

LightningModule - PyTorch Lightning - Read the Docs
A LightningModule organizes your PyTorch code into 6 sections: ... put model in train mode and enable gradient calculation model.train() ...
Read more >
Image Recognition Using Pytorch Lightning - Analytics Vidhya
The dataset has two main folders “Train” and “Test” that each contains 5 sub-folders the 5 sub-folders contain Images and the class of...
Read more >
How To Train SegFormer on a Custom Dataset - Roboflow Blog
In this post, we will walk through how to train SegFormer on a custom dataset using Pytorch Lightning to classify every pixel in...
Read more >
Adding images to your dataset - Amazon Lookout for Vision
You can add more images to your datasets by uploading images from your local computer. To add more labeled images with the SDK,...
Read more >
EfficientDet Meets Pytorch-Lightning
Computer vision went through a fast cycle of innovations and ... There are few images in the test dataset that aren't 1024 x...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found