question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

System information

  • Have I written custom code (as opposed to using a stock example script provided in MLflow): custom model training script
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04 in Docker container
  • MLflow installed from (source or binary): source (mlflow repository, master branch)
  • MLflow version (run mlflow --version): 1.3.1.dev0
  • Python version: 3.7.3
  • npm version, if running the dev UI: 3.10.10
  • Exact command to reproduce: docker-compose up

Describe the problem

  1. On open some run in mlflow ui:
2019/10/22 10:20:08 ERROR mlflow.server: Exception on /ajax-api/2.0/preview/mlflow/model-versions/search [GET]
 Traceback (most recent call last):
   File "/usr/local/lib/python3.7/site-packages/Flask-1.1.1-py3.7.egg/flask/app.py", line 2446, in wsgi_app
     response = self.full_dispatch_request()
   File "/usr/local/lib/python3.7/site-packages/Flask-1.1.1-py3.7.egg/flask/app.py", line 1951, in full_dispatch_request
     rv = self.handle_user_exception(e)
   File "/usr/local/lib/python3.7/site-packages/Flask-1.1.1-py3.7.egg/flask/app.py", line 1820, in handle_user_exception
     reraise(exc_type, exc_value, tb)
   File "/usr/local/lib/python3.7/site-packages/Flask-1.1.1-py3.7.egg/flask/_compat.py", line 39, in reraise
     raise value
   File "/usr/local/lib/python3.7/site-packages/Flask-1.1.1-py3.7.egg/flask/app.py", line 1949, in full_dispatch_request
     rv = self.dispatch_request()
   File "/usr/local/lib/python3.7/site-packages/Flask-1.1.1-py3.7.egg/flask/app.py", line 1935, in dispatch_request
     return self.view_functions[rule.endpoint](**req.view_args)
   File "/usr/local/lib/python3.7/site-packages/mlflow-1.3.1.dev0-py3.7.egg/mlflow/server/handlers.py", line 137, in wrapper
     return func(*args, **kwargs)
   File "/usr/local/lib/python3.7/site-packages/mlflow-1.3.1.dev0-py3.7.egg/mlflow/server/handlers.py", line 581, in _search_model_versions
     model_versions_detailed = _get_model_registry_store().search_model_versions(
 AttributeError: 'NoneType' object has no attribute 'search_model_versions'
  1. On click Register Model:

mlflow_model_registry_error

Code to reproduce issue

Project sources mlflow_model_registry.zip

Archive content:

I. Dockerfile

FROM python
WORKDIR /home

# RUN pip install sklearn git+https://github.com/mparkhe/mlflow.git@ssue-19
RUN apt-get update && \
    curl -sL https://deb.nodesource.com/setup_6.x | bash - && \
    apt-get update && \
    apt-get install -y nodejs npm

RUN pip install sklearn

RUN git clone https://github.com/mlflow/mlflow && \
    cd mlflow && python setup.py install && \
    cd mlflow/server/js && npm install

ENV MLFLOW_TRACKING_URI=/home/mlruns

CMD mlflow server --host 0.0.0.0 --port 5000 & (cd mlflow/mlflow/server/js && npm start)

II. docker-compose.yml

version: '3'

services:
  mlflow_model_registry:
    build:
      context: .

    image: mlflow_model_registry:latest

    container_name: mlflow_model_registry_1

    ports:
      - 5000:5000
      - 3000:3000

    volumes:
      - ./main.py:/home/main.py
      - ./mlruns:/home/mlruns

III. main.py

import joblib
import mlflow
from mlflow.sklearn import log_model
from sklearn import svm
from sklearn import datasets


clf = svm.SVC(gamma='scale')
iris = datasets.load_iris()
X, y = iris.data, iris.target
clf.fit(X, y)

joblib.dump(clf, 'iris.model')

log_model('iris.model', 'model')

Run all:

  1. build image and run container
docker-compose build
docker-compose up
  1. enter docker container in new terminal window
docker exec -ti mlflow_model_registry_1 /bin/bash
  1. run training script
python main.py
  1. open dev UI

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:18 (7 by maintainers)

github_iconTop GitHub Comments

4reactions
SteNicholascommented, Dec 3, 2019

@MeisterUrian I ran into the exact same issue. This is because Model Registry does not support file based stores yet. (https://github.com/mlflow/mlflow/blob/master/mlflow/server/handlers.py#L60)

I used my local PostgreSQL as a tracking server and it worked.

@harupy When does Model Registry support file based stores? And MLflow Model Registry doesn’t mention about this. It is necessary to tell the truth for new users that Model Registry doesn’t support file based stores.

4reactions
kadakiacommented, Nov 5, 2019

@harupy Should model registry work if the tracking server is on an ec2 instance (with artifacts being logged to s3)? Because I’m running into the same issue, it seems.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Bug (2006) - IMDb
An unhinged war veteran holes up with a lonely woman in a spooky Oklahoma motel room. The line between reality and delusion is...
Read more >
bug - Wiktionary
(entomology) An insect of the order Hemiptera (the “true bugs”). Any of various species of marine or freshwater crustaceans; e.g. a Moreton Bay...
Read more >
Bug Definition & Meaning - Merriam-Webster
The meaning of BUG is any of an order (Hemiptera and especially its suborder Heteroptera) of insects (such as an assassin bug or...
Read more >
Bug - Wikipedia
A terrestrial arthropod animal (with at least six legs). Insect, a six-legged arthropod · Covert listening device, used in surveillance, espionage and policing ......
Read more >
BUG | definition in the Cambridge English Dictionary
an insect: Some tiny white bugs had eaten the leaves of my house plants. ... A bug is also a small organism that...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found