[BUG] Can't replicate quickstart save and serve example

Willingness to contribute

No. I cannot contribute a bug fix at this time.

MLflow version

1.27.0

System information

• Windows 11
• Python 3.9.7
• conda 4.13.0

Describe the problem

I’m trying to replicate the quickstart save and serve example. I go to the example folder, run the python script and can see the model runs and artifacts when I type “mlflow ui”. However, when I try the mlflow serve command with different model run Ids and ports I get a 404 in my browser, even though the command seems successful:

mlflow models serve -m runs:/e1dabe8fc6e84286af5bee28ca89cdde/model --port 1234
2022/07/11 07:40:01 INFO mlflow.models.cli: Selected backend for flavor 'python_function'
2022/07/11 07:40:02 INFO mlflow.utils.conda: Conda environment mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4 already exists.
2022/07/11 07:40:02 INFO mlflow.pyfunc.backend: === Running command 'conda activate mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4 & waitress-serve --host=127.0.0.1 --port=1234 --ident=mlflow mlflow.pyfunc.scoring_server.wsgi:app'
INFO:waitress:Serving on http://127.0.0.1:1234

I tried running directly from anaconda prompt, and I get the following error:

conda activate mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4 & waitress-serve --host=127.0.0.1 --port=1234 --ident=mlflow mlflow.pyfunc.scoring_server.wsgi:app

Traceback (most recent call last):
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\Scripts\waitress-serve.exe\__main__.py", line 7, in <module>
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\site-packages\waitress\runner.py", line 283, in run
    app = resolve(module, obj_name)
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\site-packages\waitress\runner.py", line 218, in resolve
    obj = __import__(module_name, fromlist=segments[:1])
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\site-packages\mlflow\pyfunc\scoring_server\wsgi.py", line 6, in <module>
    app = scoring_server.init(load_model(os.environ[scoring_server._SERVER_MODEL_PATH]))
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\os.py", line 679, in __getitem__
    raise KeyError(key) from None
KeyError: '__pyfunc_model_path__'

I have tried deleting and creating a new anaconda environment, ran from git bash, anaconda prompt, added anaconda3 environment variables. I know it has something to do with the _SERVER_MODEL_PATH variable but I wouldn’t know how to set it up or which path add to my environment variables so it can read this variable from there.

Please help,

Sergio

Tracking information

mlflow models serve -m runs:/e1dabe8fc6e84286af5bee28ca89cdde/model --port 1234
2022/07/11 07:40:01 INFO mlflow.models.cli: Selected backend for flavor 'python_function'
2022/07/11 07:40:02 INFO mlflow.utils.conda: Conda environment mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4 already exists.
2022/07/11 07:40:02 INFO mlflow.pyfunc.backend: === Running command 'conda activate mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4 & waitress-serve --host=127.0.0.1 --port=1234 --ident=mlflow mlflow.pyfunc.scoring_server.wsgi:app'
INFO:waitress:Serving on http://127.0.0.1:1234

I tried running directly from anaconda prompt, and I get the following error:

conda activate mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4 & waitress-serve --host=127.0.0.1 --port=1234 --ident=mlflow mlflow.pyfunc.scoring_server.wsgi:app

Traceback (most recent call last):
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\Scripts\waitress-serve.exe\__main__.py", line 7, in <module>
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\site-packages\waitress\runner.py", line 283, in run
    app = resolve(module, obj_name)
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\site-packages\waitress\runner.py", line 218, in resolve
    obj = __import__(module_name, fromlist=segments[:1])
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\site-packages\mlflow\pyfunc\scoring_server\wsgi.py", line 6, in <module>
    app = scoring_server.init(load_model(os.environ[scoring_server._SERVER_MODEL_PATH]))
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\os.py", line 679, in __getitem__
    raise KeyError(key) from None
KeyError: '__pyfunc_model_path__'

Code to reproduce issue

mlflow models serve -m runs:/e1dabe8fc6e84286af5bee28ca89cdde/model --port 1234
2022/07/11 07:40:01 INFO mlflow.models.cli: Selected backend for flavor 'python_function'
2022/07/11 07:40:02 INFO mlflow.utils.conda: Conda environment mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4 already exists.
2022/07/11 07:40:02 INFO mlflow.pyfunc.backend: === Running command 'conda activate mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4 & waitress-serve --host=127.0.0.1 --port=1234 --ident=mlflow mlflow.pyfunc.scoring_server.wsgi:app'
INFO:waitress:Serving on http://127.0.0.1:1234

#I tried running directly from anaconda prompt, and I get the following error:

conda activate mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4 & waitress-serve --host=127.0.0.1 --port=1234 --ident=mlflow mlflow.pyfunc.scoring_server.wsgi:app

Traceback (most recent call last):
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\Scripts\waitress-serve.exe\__main__.py", line 7, in <module>
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\site-packages\waitress\runner.py", line 283, in run
    app = resolve(module, obj_name)
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\site-packages\waitress\runner.py", line 218, in resolve
    obj = __import__(module_name, fromlist=segments[:1])
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\site-packages\mlflow\pyfunc\scoring_server\wsgi.py", line 6, in <module>
    app = scoring_server.init(load_model(os.environ[scoring_server._SERVER_MODEL_PATH]))
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\os.py", line 679, in __getitem__
    raise KeyError(key) from None
KeyError: '__pyfunc_model_path__'

Other info / logs

mlflow models serve -m runs:/e1dabe8fc6e84286af5bee28ca89cdde/model --port 1234
2022/07/11 07:40:01 INFO mlflow.models.cli: Selected backend for flavor 'python_function'
2022/07/11 07:40:02 INFO mlflow.utils.conda: Conda environment mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4 already exists.
2022/07/11 07:40:02 INFO mlflow.pyfunc.backend: === Running command 'conda activate mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4 & waitress-serve --host=127.0.0.1 --port=1234 --ident=mlflow mlflow.pyfunc.scoring_server.wsgi:app'
INFO:waitress:Serving on http://127.0.0.1:1234

I tried running directly from anaconda prompt, and I get the following error:

conda activate mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4 & waitress-serve --host=127.0.0.1 --port=1234 --ident=mlflow mlflow.pyfunc.scoring_server.wsgi:app

Traceback (most recent call last):
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\Scripts\waitress-serve.exe\__main__.py", line 7, in <module>
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\site-packages\waitress\runner.py", line 283, in run
    app = resolve(module, obj_name)
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\site-packages\waitress\runner.py", line 218, in resolve
    obj = __import__(module_name, fromlist=segments[:1])
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\site-packages\mlflow\pyfunc\scoring_server\wsgi.py", line 6, in <module>
    app = scoring_server.init(load_model(os.environ[scoring_server._SERVER_MODEL_PATH]))
  File "C:\Users\sergio ferro\.conda\envs\mlflow-ddf4db606beaa0e9bb42ff0ed98e8f4c4c7cb1f4\lib\os.py", line 679, in __getitem__
    raise KeyError(key) from None
KeyError: '__pyfunc_model_path__'

What component(s) does this bug affect?

• area/artifacts: Artifact stores and artifact logging
• area/build: Build and test infrastructure for MLflow
• area/docs: MLflow documentation pages
• area/examples: Example code
• area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
• area/models: MLmodel format, model serialization/deserialization, flavors
• area/pipelines: Pipelines, Pipeline APIs, Pipeline configs, Pipeline Templates
• area/projects: MLproject format, project running backends
• area/scoring: MLflow Model server, model deployment tools, Spark UDFs
• area/server-infra: MLflow Tracking server backend
• area/tracking: Tracking Service, tracking client APIs, autologging

What interface(s) does this bug affect?

• area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
• area/docker: Docker use across MLflow’s components, such as MLflow Projects and MLflow Models
• area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
• area/windows: Windows support

What language(s) does this bug affect?

• language/r: R APIs and clients
• language/java: Java APIs and clients
• language/new: Proposals for new client languages

What integration(s) does this bug affect?

• integrations/azure: Azure and Azure ML integrations
• integrations/sagemaker: SageMaker integrations
• integrations/databricks: Databricks integrations

[smferro54]

Oh gosh this was very silly of me, thanks a lot for your help and patience! Best, Sergio

On Mon, Jul 11, 2022, 09:33 Harutaka Kawamura @.***> wrote:

As explained in https://mlflow.org/docs/latest/quickstart.html#saving-and-serving-models , http://127.0.0.1:5001/ is the endpoint for inference.

— Reply to this email directly, view it on GitHub https://github.com/mlflow/mlflow/issues/6226#issuecomment-1180415873, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHD4HVYYLOFNV4O5WFT3BEDVTQPBTANCNFSM53HIKSIQ . You are receiving this because you were mentioned.Message ID: @.***>

[harupy]

Glad to help!