Add image vision datasets as Base lightningModules
See original GitHub issueAdd the torchvision datasets including the standard transforms and splits as modules
class MNISTModule(pl.LightningModule):
def prepare_data(self):
self.mnist_train = MNIST(os.getcwd(), train=True, download=True, transform=transforms.ToTensor())
self.mnist_test = MNIST(os.getcwd(), train=False, download=True, transform=transforms.ToTensor())
def train_dataloader(self):
loader = DataLoader(self.mnist_train, batch_size=self.hparams.batch_size, num_workers=self.hparams.num_workers)
return loader
def val_dataloader(self):
loader = DataLoader(self.mnist_test, batch_size=self.hparams.batch_size, num_workers=self.hparams.num_workers)
return loader
def test_dataloader(self):
loader = DataLoader(self.mnist_test, batch_size=self.hparams.batch_size, num_workers=self.hparams.num_workers)
return loader
from pytorch_lightning.bolts.dataset_modules import MNISTModule
class MyModel(MNISTModule, pl.LightningModule):
# all the dataloaders have been implemented with MNIST
Issue Analytics
- State:
- Created 3 years ago
- Reactions:1
- Comments:10 (7 by maintainers)
Top Results From Across the Web
LightningModule - PyTorch Lightning - Read the Docs
A LightningModule organizes your PyTorch code into 6 sections: ... put model in train mode and enable gradient calculation model.train() ...
Read more >Image Recognition Using Pytorch Lightning - Analytics Vidhya
The dataset has two main folders “Train” and “Test” that each contains 5 sub-folders the 5 sub-folders contain Images and the class of...
Read more >How To Train SegFormer on a Custom Dataset - Roboflow Blog
In this post, we will walk through how to train SegFormer on a custom dataset using Pytorch Lightning to classify every pixel in...
Read more >Adding images to your dataset - Amazon Lookout for Vision
You can add more images to your datasets by uploading images from your local computer. To add more labeled images with the SDK,...
Read more >EfficientDet Meets Pytorch-Lightning
Computer vision went through a fast cycle of innovations and ... There are few images in the test dataset that aren't 1024 x...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I agree with @williamFalcon to only include curated datasets and potentially all from torchvision since it has a quite robust API. In case of missing ones, I think it’s best to invest in contributing to torchvision instead of start creating custom stuff. For downloading, I would give the control to the user. Not all of the datasets in torchvision have really this option since in some of them you have to download the data in an offline process.
we need to evaluate direct support with PL after we test this for a bit