[Azure ML] score.py not being copied to container using SDK v2
See original GitHub issue- Package Name: azure-ai-ml
- Package Version: 0.1.0b5
- Operating System: Tested on both windows and macos
- Python Version: 3.9
Describe the bug
score.py
is not being copied to a container, tried both relative and absolute path. Due to which container fails at runtime with ModuleNotFoundError: No module named 'main'
To Reproduce Steps to reproduce the behavior: Sample code is given below:
env = Environment(
conda_file="./code/deploy/environment.yml",
image="mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20210727.v1",
)
blue_deployment = ManagedOnlineDeployment(
name="blue",
endpoint_name=local_endpoint_name,
model=model,
environment=env,
code_configuration=CodeConfiguration(
code="./code/deploy", scoring_script="score.py"
),
instance_type="Standard_F2s_v2",
instance_count=1,
)
And directory listing from inside the docker container:
root@21d33d6d1768:/var/azureml-app# ls -alih
total 16K
714328 drwxr-xr-x 1 root root 4.0K Jul 22 12:52 .
714327 drwxr-xr-x 1 root root 4.0K Jul 22 12:19 ..
714329 drwxr-xr-x 2 root root 4.0K Jul 22 12:47 .empty
570216 -rw-r--r-- 1 root root 259 Jul 22 12:18 conda.yml
root@21d33d6d1768:/var/azureml-app#
And exception logs when container runs
(ewa_deployer_v2) waqas@192-168-1-13 EWA.CreditRisk.Model % docker run local-07222036358655:blue
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
2022-07-22T15:15:25,827211584+00:00 - gunicorn/run
2022-07-22T15:15:25,847585459+00:00 - rsyslog/run
2022-07-22T15:15:25,849390709+00:00 - iot-server/run
2022-07-22T15:15:25,877277918+00:00 - nginx/run
EdgeHubConnectionString and IOTEDGE_IOTHUBHOSTNAME are not set. Exiting...
2022-07-22T15:15:26,660859376+00:00 - iot-server/finish 1 0
2022-07-22T15:15:26,682624043+00:00 - Exit code 1 is normal. Not restarting iot-server.
Dynamic Python package installation is disabled.
Starting HTTP server
Starting gunicorn 20.1.0
Listening at: http://127.0.0.1:31311 (64)
Using worker: sync
worker timeout is set to 300
Booting worker with pid: 152
SPARK_HOME not set. Skipping PySpark Initialization.
Exception in worker process
Traceback (most recent call last):
File "/opt/miniconda/envs/inf-conda-env/lib/python3.9/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
worker.init_process()
File "/opt/miniconda/envs/inf-conda-env/lib/python3.9/site-packages/gunicorn/workers/base.py", line 134, in init_process
self.load_wsgi()
File "/opt/miniconda/envs/inf-conda-env/lib/python3.9/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
self.wsgi = self.app.wsgi()
File "/opt/miniconda/envs/inf-conda-env/lib/python3.9/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/opt/miniconda/envs/inf-conda-env/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
return self.load_wsgiapp()
File "/opt/miniconda/envs/inf-conda-env/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
return util.import_app(self.app_uri)
File "/opt/miniconda/envs/inf-conda-env/lib/python3.9/site-packages/gunicorn/util.py", line 359, in import_app
mod = importlib.import_module(module)
File "/opt/miniconda/envs/inf-conda-env/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/var/azureml-server/entry.py", line 1, in <module>
import create_app
File "/var/azureml-server/create_app.py", line 4, in <module>
from routes_common import main
File "/var/azureml-server/routes_common.py", line 32, in <module>
from aml_blueprint import AMLBlueprint
File "/var/azureml-server/aml_blueprint.py", line 30, in <module>
import main
ModuleNotFoundError: No module named 'main'
Worker exiting (pid: 152)
Shutting down: Master
Reason: Worker failed to boot.
Expected behavior scoring file should be copied and container starts without any issue
Issue Analytics
- State:
- Created a year ago
- Comments:8 (3 by maintainers)
Top Results From Across the Web
Troubleshooting online endpoints deployment and scoring
Learn how to troubleshoot some common deployment and scoring errors with online endpoints.
Read more >Manage Azure Machine Learning environments with the CLI ...
Learn how to manage Azure ML environments using Python SDK and Azure CLI extension for Machine Learning.
Read more >Tutorial: ML pipelines with Python SDK v2 - Azure
In this tutorial, you'll use Azure Machine Learning (Azure ML) to create a production ready machine learning (ML) project, using AzureML Python ......
Read more >Advanced entry script authoring - Azure - Microsoft Learn
Azure Machine Learning then creates an OpenAPI (Swagger) specification for the web service during deployment. Warning. You must not use ...
Read more >Access data in a job - Azure Machine Learning | Microsoft Learn
Learn how to read and write data for your jobs with the Azure Machine Learning Python SDK v2 and the Azure Machine Learning...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Is there any solution or associated root cause of this issue. I am experiencing the same when using the azure CLI v2 for deployment
@waqassiddiqi could I receive the environment.yaml and score.py file used. Also, what region was this done in?