conda environment installation takes many hours

More of a conda issue than an atac-seq-pipeline issue, but wondering if the dev team has any suggestions.

running ./install_conda_env.sh takes many hours, stuck at solving environment step:

=== Installing pipeline's Conda environments ===
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working...

(after ~6 hours the installation is successful)

I have tried the following to resolve the issue:

• setting channel priority to strict / flexible/ false – using ‘strict’ priority gives package conflicts; flexible/false take a long time to resolve.

• removing some of the strict package version specifications in requirements.txt (i.e. using a newer package version if available) – doesn’t seem to affect install speed

• removing the ‘defaults’ conda channel from this line:

conda create -n ${CONDA_ENV_PY3} --file ${REQ_TXT_PY3} -y -c defaults -c r -c bioconda -c conda-forge

since the Iinternet seems to suggest that the defaults conda channel can be problematic.

I’m using conda 4.9.2

Thanks so much for any insights/advise!

[leepc12]

Thanks @leipzig Added mamba to pipeline’s Conda installer. It’s like 5 times faster than Conda.

[leepc12]

If both mamba or no-mamba methods don’t work then please use Docker (caper run/submit ... --docker). If your cluster doesn’t allow it then try with Singularity (caper run/submit ... --singularity). Check if you have Singularity on your system.