Distribute Lite's code with PyTorch Lightning

Proposed refactor

Remove this line https://github.com/Lightning-AI/lightning/blob/5025c3ec162b7a1af978bfb19424fd1bb2683012/requirements/pytorch/base.txt#L6

and distribute the Lite source code together with PL when it’s installed

Motivation

When you do

PACKAGE_NAME=pytorch pip install -e .

to create an editable installation of PL. Lite gets installed from PyPI:

$ pip freeze | grep lightning
lightning-lite==1.8.0.post1
-e git+https://github.com/Lightning-AI/lightning.git@5025c3ec162b7a1af978bfb19424fd1bb2683012#egg=pytorch_lightning
...

This means that local changes to Lite are not reflected when developing locally

One downside of this is that if you also do pip install lightning_lite, you might have conflicting lite installations. The PyPI version will take precedence.

---

If you enjoy Lightning, check out our other projects! ⚡

• Metrics: Machine learning metrics for distributed, scalable PyTorch applications.

• Lite: enables pure PyTorch users to scale their existing code on any kind of device while retaining full control over their own loops and optimization logic.

• Flash: The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, fine-tuning, and solving problems with deep learning.

• Bolts: Pretrained SOTA Deep Learning models, callbacks, and more for research and production with PyTorch Lightning and PyTorch.

• Lightning Transformers: Flexible interface for high-performance research using SOTA Transformers leveraging PyTorch Lightning, Transformers, and Hydra.

cc @carmocca @akihironitta @borda

[carmocca]

That happens because the docker image used already contains PyPI’s lighting_lite installation. And given the issue described above:

One downside of this is that if you also do pip install lightning_lite, you might have conflicting lite installations. The PyPI version will take precedence.

The PyPI version is used over the version packaged inside PL.

Let’s give it a day to see if the rebuilt docker images do not have it installed which would make this issue disappear.

[carmocca]

That’s weird! I will have a look. Thanks for reporting