[BUG] Conda install takes...forever?

Bug description

Ran conda install -c nvidia -c conda-forge transformers4rec on an Ubuntu 20.04 machine. The install runs for an unreasonable amount of time, I haven’t seen it end yet. I will try to let a run go on as long as possible to see if it ever ends. I’ve also tried targeting specific versions e.g. transformers4rec=0.1.4 and the same issue occurs.

Steps/Code to reproduce bug

1. Create conda environment on Ubuntu 20.04 machine
2. Run conda install -c nvidia -c conda-forge transformers4rec
3. Notice the install runs for a very long time.

Expected behavior

Install to take less than a few minutes, and for transformers4rec to be available to start using in the conda environment.

Environment details

• Transformers4Rec version: 0.1.4 (attempted)
• Platform: Ubuntu 20.04
• Python version: 3.8
• Huggingface Transformers version: X
• PyTorch version (GPU?): X
• Tensorflow version (GPU?): X

Additional context

At the start of the install I also see:

Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.

Then Solving environment: - for a long time.

[bschifferer]

Hello @masoncusack,

thanks for sharing. We will work on it to provide better installation instructions.

Currently, I can provide you some “workarounds” (for GPU support):

1. Installing Conda in two steps: I think the main issue in freezing the installation is to resolve dependencies of RAPIDs and Transformer4Rec. We use RAPIDs cuDF and dask_cudf. I was able to install Transformer4Rec by installing RAPIDs, first and then Transformer4Rec.

conda create -y -n rapids -c rapidsai -c nvidia -c conda-forge rapids=22.06 python=3.8 cudatoolkit=11.2
### Activate the environment
conda install -c nvidia -c conda-forge transformers4rec[pytorch,nvtabular]

Transformer4Rec (and NVTabular) are based on cuDF/RAPIDS 22.06 release. You need to have the CUDA and supporting CUDA libraries installed. You can install the libraries via gitclone and pip inside the conda environment. This is often faster than resolving and installing dependencies

conda create -y -n rapids -c rapidsai -c nvidia -c conda-forge rapids=22.06 python=3.8 cudatoolkit=11.2
### Activate the environment
git clone https://github.com/NVIDIA-Merlin/Transformers4Rec.git && cd Transformers4Rec && pip install . && cd ..
git clone https://github.com/NVIDIA-Merlin/NVTabular.git && cd NVTabular && pip install . && cd ..
git clone https://github.com/NVIDIA-Merlin/dataloader.git && cd dataloader && pip install . && cd ..
git clone https://github.com/NVIDIA-Merlin/core.git && cd core && pip install . && cd ..

Do you use a specific docker container that I can reproduce the installation end-to-end?

[masoncusack]

Hi @bschifferer 👋 It’s a new conda environment in that there’s nothing else installed there.

I just set up the environment with

conda create -n transformers4rec python=3.8 && conda activate transformers4rec