Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

azureml-core cannot be installed in custom image for AML

See original GitHub issue

Package Name: azureml-core, azureml-pipeline, azureml-pipeline-steps, azureml-pipeline-core
Package Version: latest
Operating System: Ubuntu 20.04
Python Version: 3.8.13

Describe the bug I tried to build a custom image on Azure Machine Learning, with a base image from mcr.microsoft.com/azureml/openmpi4.1.0-cuda11.3-cudnn8-ubuntu20.04 and additional conda specifications:

name: project_environment
dependencies:
  - python=3.8.13
  - pip:
      - azureml==0.2.7
      - azureml-dataprep==4.0.4
      - datasets==2.3.2
      - packaging==21.3
      - pandas==1.4.2
      - psutil==5.9.1
      - pytorch_lightning==1.6.4
      - PyYAML==6.0
      - setuptools==61.2.0
      - torch==1.11.0
      - transformers==4.11.3
  - pip
channels:
  - anaconda
  - conda-forge

The build succeeded. However, when I tried to add the following packages to the pip list:

      - azureml-core==1.43.0
      - azureml-pipeline==1.43.0
      - azureml-pipeline-steps==1.43.0
      - azureml-pipeline-core==1.43.0

The build status is stuck at “Running” for 1.5 hours and then timed out. The build log shows it was stuck at the installing pip dependencies step:

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Installing pip dependencies: ...working... 
Run ID: cxp timed out after 1h30m0s

To Reproduce Steps to reproduce the behavior:

Prepare the following conda-dependencies.yaml:

name: project_environment
dependencies:
  - python=3.8.13
  - pip:
      - azureml==0.2.7
      - azureml-core==1.43.0
      - azureml-pipeline==1.43.0
      - azureml-pipeline-steps==1.43.0
      - azureml-pipeline-core==1.43.0
      - datasets==2.3.2
      - packaging==21.3
      - pandas==1.4.2
      - psutil==5.9.1
      - pytorch_lightning==1.6.4
      - PyYAML==6.0
      - setuptools==61.2.0
      - torch==1.11.0
      - transformers==4.11.3
  - pip
channels:
  - anaconda
  - conda-forge

Set up a custom AML image:

    env = Environment.from_conda_specification(
            name=env_name,
            file_path="conda-dependencies.yaml")
    env.docker.base_image = 'mcr.microsoft.com/azureml/openmpi4.1.0-cuda11.3-cudnn8-ubuntu20.04'
    env.register(ws)

Expected behavior If the pip packages not found because of some reasons, the build should fail with an error message instead of hanging. I do not see any reason why azureml-core et al. should cause problems though. They are public PyPi packages.

Screenshots