Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

DeepSpeed initialization with GNN-like model

See original GitHub issue

My code is quite similar to some GNN structure : NN_output = graph.forward(NN_input, types=“f”)

So, outputs = model_engine(inputs) seems does not really fit in my case ? args also does not follow such code styling.

Any idea ?

Issue Analytics

State:
Created a year ago
Comments:20 (9 by maintainers)

Top GitHub Comments

1reaction

tjruwasecommented, Jul 30, 2022

@buttercutter, #1149 is now merged. Please try master.

1reaction

tjruwasecommented, Jul 13, 2022

Set “train_micro_batch_size_per_gpu” to 8 in the configuration file.

Top Results From Across the Web

Training Setup — DeepSpeed 0.8.0 documentation

The entrypoint for all training with DeepSpeed is deepspeed.initialize() . Will initialize distributed backend if it is not initialized already.

Getting Started - DeepSpeed

DeepSpeed model training is accomplished using the DeepSpeed engine. ... deepspeed.initialize ensures that all of the necessary setup ...

DeepSpeed Integration - Hugging Face

i.e. you will need to re-initialize the deepspeed engine, since model.load_state_dict(state_dict) will remove all the DeepSpeed magic from it. So do this only ......

DeepSpeed - Microsoft Research

DeepSpeed, part of Microsoft AI at Scale, is a deep learning optimization library that makes distributed training ... Reshape Large Model Training Landscape....

Enabling Efficient Inference of Transformer Models at ... - arXiv

high inference throughput with large models which do not fit in aggregate GPU memory. DeepSpeed Inference reduces latency by up to 7.3× over....