Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

pip install with building ops succeeds without installing any ops.

See original GitHub issue

I’m trying to run deepseed with cpu adam, and after doing:

DS_BUILD_OPS=1 pip install deepspeed

here the output of ds_report:

--------------------------------------------------
DeepSpeed C++/CUDA extension op report
--------------------------------------------------
NOTE: Ops not installed will be just-in-time (JIT) compiled at
      runtime if needed. Op compatibility means that your system
      meet the required dependencies to JIT install the op.
--------------------------------------------------
JIT compiled ops requires ninja
ninja .................. [OKAY]
--------------------------------------------------
op name ................ installed .. compatible
--------------------------------------------------
cpu_adam ............... [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
fused_lamb ............. [NO] ....... [OKAY]
/bin/sh: line 1: type: llvm-config-9: not found
/bin/sh: line 1: type: cmake: not found
 [WARNING]  sparse_attn requires the 'cmake' command, but it does not exist!
sparse_attn ............ [NO] ....... [NO]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
utils .................. [NO] ....... [OKAY]
 [WARNING]  async_io requires the libraries: ['libaio-dev'] but are missing.
async_io ............... [NO] ....... [NO]
--------------------------------------------------
DeepSpeed general environment info:
torch install path ............... ['/home/sgugger/.pyenv/versions/3.7.9/envs/base/lib/python3.7/site-packages/torch']
torch version .................... 1.8.1+cu102
torch cuda version ............... 10.2
nvcc version ..................... 10.2
deepspeed install path ........... ['/home/sgugger/.pyenv/versions/3.7.9/envs/base/lib/python3.7/site-packages/deepspeed']
deepspeed info ................... 0.3.16, unknown, unknown
deepspeed wheel compiled w. ...... torch 1.8, cuda 10.2

I then cannot run anything using cpu adam (it’s not JITted at runtime as implied by the report), for instance pytest tests/deepspeed/test_deepspeed.py in the Transformers repo fails.

If you have any insight on how to get those ops working, that would be great!