[BUG]

Describe the bug

Attempting to install and run pip release v1.1.0, unable to execute annotate subcommand - I eventually arrive at the following errors:

FileNotFoundError: [Errno 2] No such file or directory: 'mlflow' and ModuleNotFoundError: No module named 'scanpy' after installing cellxgene v1.1.0 in a fresh conda environment

To Reproduce Steps to reproduce the behavior:

commands:

conda create -n "annoTEST2" python=3.9
conda activate annoTEST2
pip install cellxgene
cellxgene annotate -h
cellxgene annotate -i pbmc3k.h5ad -m immune_ref_20220728145305_conda.zip -o pbmc3kAnnotated.h5ad

output:

(annoTEST2) mlombardo@CZIMACOS3301 test % cellxgene annotate -i pbmc3k.h5ad -m immune_ref_20220728145305_conda.zip -o pbmc3kAnnotated.h5ad
Reading query dataset pbmc3k.h5ad...
Using cached model at .models_cache/immune_ref_20220728145305_conda
Annotating pbmc3k.h5ad with cell_type...
Traceback (most recent call last):
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/bin/cellxgene", line 8, in <module>
    sys.exit(cli())
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/server/cli/annotate.py", line 189, in annotate
    p = subprocess.Popen(
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/subprocess.py", line 1821, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'mlflow'

I install mlflow and run the same annotate command, and get the following error:

(annoTEST2) mlombardo@CZIMACOS3301 test % cellxgene annotate -i pbmc3k.h5ad -m immune_ref_20220728145305_conda.zip -o pbmc3kAnnotated.h5ad
Reading query dataset pbmc3k.h5ad...
Using cached model at .models_cache/immune_ref_20220728145305_conda
Annotating pbmc3k.h5ad with cell_type...
/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/click/core.py:2322: UserWarning: Virtualenv support is still experimental and may be changed in a future release without warning.
  value = self.callback(ctx, self, value)
2022/09/09 00:07:34 INFO mlflow.models.cli: Selected backend for flavor 'python_function'
2022/09/09 00:07:34 INFO mlflow.utils.virtualenv: Installing python 3.9.13 if it does not exist
2022/09/09 00:07:34 INFO mlflow.utils.virtualenv: Environment /Users/mlombardo/.mlflow/envs/mlflow-155420bfe319af3b067b955e140aacfc86c2669e already exists
2022/09/09 00:07:34 INFO mlflow.utils.virtualenv: Running command: bash -c source /Users/mlombardo/.mlflow/envs/mlflow-155420bfe319af3b067b955e140aacfc86c2669e/bin/activate && python -c "from mlflow.pyfunc.scoring_server import _predict; _predict(model_uri='file:///Users/mlombardo/Downloads/test/.models_cache/immune_ref_20220728145305_conda', input_path='/var/folders/1v/yhtk99j50v351lg25mn8yz200000gq/T/tmpx4xmnf_7', output_path=None, content_type='csv', json_format='split')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/mlombardo/.mlflow/envs/mlflow-155420bfe319af3b067b955e140aacfc86c2669e/lib/python3.9/site-packages/mlflow/pyfunc/scoring_server/__init__.py", line 330, in _predict
    pyfunc_model = load_model(model_uri)
  File "/Users/mlombardo/.mlflow/envs/mlflow-155420bfe319af3b067b955e140aacfc86c2669e/lib/python3.9/site-packages/mlflow/pyfunc/__init__.py", line 756, in load_model
    model_impl = importlib.import_module(conf[MAIN])._load_pyfunc(data_path)
  File "/Users/mlombardo/.pyenv/versions/3.9.13/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/Users/mlombardo/Downloads/test/.models_cache/immune_ref_20220728145305_conda/code/cell_type/mlflow_model_loader/__init__.py", line 1, in <module>
    from cell_type.mlflow_model_loader.model_wrapper import CellTypeAnnotationModelWrapper
  File "/Users/mlombardo/Downloads/test/.models_cache/immune_ref_20220728145305_conda/code/cell_type/mlflow_model_loader/model_wrapper.py", line 3, in <module>
    import scanpy
ModuleNotFoundError: No module named 'scanpy'
Traceback (most recent call last):
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/bin/mlflow", line 8, in <module>
    sys.exit(cli())
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/mlflow/models/cli.py", line 125, in predict
    return _get_flavor_backend(
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/mlflow/pyfunc/backend.py", line 133, in predict
    return _execute_in_virtualenv(
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/mlflow/utils/virtualenv.py", line 338, in _execute_in_virtualenv
    return _exec_cmd(
  File "/Users/mlombardo/opt/anaconda3/envs/annoTEST2/lib/python3.9/site-packages/mlflow/utils/process.py", line 116, in _exec_cmd
    raise ShellCommandException.from_completed_process(comp_process)
mlflow.utils.process.ShellCommandException: Non-zero exit code: 1
Command: ['bash', '-c', 'source /Users/mlombardo/.mlflow/envs/mlflow-155420bfe319af3b067b955e140aacfc86c2669e/bin/activate && python -c "from mlflow.pyfunc.scoring_server import _predict; _predict(model_uri=\'file:///Users/mlombardo/Downloads/test/.models_cache/immune_ref_20220728145305_conda\', input_path=\'/var/folders/1v/yhtk99j50v351lg25mn8yz200000gq/T/tmpx4xmnf_7\', output_path=None, content_type=\'csv\', json_format=\'split\')"']
Annotation failed!

I encounter the same error even after installing scanpy

Expected behavior To run the full annotation

Screenshots If applicable, add screenshots to help explain your problem.

Version (please complete the following information):

• Desktop or hosted?: Desktop
• Browser (if hosted) [e.g. chrome, safari]:
• Version [e.g. 0.13.0]:

Additional context Add any other context about the problem here.

[MaximilianLombardo]

Ok that worked, and I also needed to specify the --no-use-gpu to get a successful run, just documenting here so that I can replicate in the docs - full command for reference:

cellxgene annotate -i airway.h5ad -m immune_ref_20220728145305_conda.zip -o airwayAnnotated.h5ad --mlflow-env-manager conda --organism 'Homo sapiens' --gene-column-name feature_name --no-use-gpu

[MaximilianLombardo]

@atolopko-czi I’m realizing that for the first issue - I needed to run it like so:

pip install 'cellxgene[annotate]'

The installation seems to have worked via that method - testing the rest of the workflow and will report back with any issues.