Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Feature Request] Add a `use_env` flag in the `deepspeed` launcher

See original GitHub issue

PyTorch provides a --use_env flag in its dsitributed launcher, and if it’s activated, does not pass --local_rank to the children processes (it sets it as an environment variable). It would be great if the DeepSpeed launcher could provide the same option so we don’t have to manually add this local_rank argument to the children processes in existing scripts.

Issue Analytics

State:
Created 2 years ago
Comments:7 (5 by maintainers)

Top GitHub Comments

1reaction

sguggercommented, Apr 4, 2022

Thanks @jeffra. Just tested locally and it works great with this flag, so closing the issue.

1reaction

jeffracommented, Apr 4, 2022

Thanks for bringing this back up @stas00, --use_env would be great. I think we actually added support for this in a slightly different flag via this PR: https://github.com/microsoft/DeepSpeed/pull/1690. It adds a few options to our launcher --no_local_rank i think would work in your case though.

Side note, I think we need to add launcher documentation to our read-the-docs page (https://deepspeed.readthedocs.io/en/latest/). Currently the only way to discover this flag would be via deepspeed --help.

Top Results From Across the Web

Training Setup — DeepSpeed 0.8.0 documentation

The set of DeepSpeed arguments include the following: 1) –deepspeed: boolean flag to enable DeepSpeed 2) –deepspeed_config <json file path>: path of a...

Implement Your First Feature Flag Tutorial - Harness

Creating and executing a Feature Flag for the React app is pretty straightforward. Depending on the language, Feature Flags can require the use ......

Feature Flagging with LaunchDarkly and Visual Studio Code

All the awesome things you can do in code don't really matter unless you have a way to actually write it. Fortunately, VS...

Distributed communication package - torch.distributed - PyTorch

We are planning on adding InfiniBand support for Gloo in the upcoming releases. ... launch a separate copy of the main training script...

Determined: Deep Learning Training Platform - PythonRepo

Feature request : it would be really helpful to be able to specify one or ... bd576678e chore: update deepspeed launcher to work...