Missing mandatory parameter --model-store

📚 The doc issue

I created a config.properties file

model_store="model_store"
load_models=all
models = {\
    "tc": {\
      "1.0.0": {\
        "defaultVersion": true,\
        "marName": "text_classifier.mar",\
        "minWorkers": 1,\
        "maxWorkers": 4,\
        "batchSize": 1,\
        "maxBatchDelay": 100,\
        "responseTimeout": 120\
      }\
    }\
  }

The documentation for torchserve states:

Customize TorchServe behaviour by using the following command line arguments when you call torchserve:

--model-store Overrides the model_store property in config.properties file
--models Overrides the load_models property in config.properties

This wording implies to me that --model-store is optional, but running torchserve --start (from a folder containing config.properties) results in the error Missing mandatory parameter --model-store

It seems to me there should only be an error if the model-store location cannot be inferred at all, i.e. it’s not passed via --model-store or defined in config.properties (it’s not clear how --model-store can ‘override’ the value in config.properties if it’s mandatory)

Suggest a potential alternative/fix

No response

[lxning]

@david-waterworth “torchserve --ncs --start” can also work if env TS_CONFIG_FILE is defined.

echo $TS_CONFIG_FILE
/Users/XXX/workplace/python_env/serve/config.properties

torchserve --ncs --start
Warning: TorchServe is using non-default JVM parameters: -Xmx4g -XX:+ExitOnOutOfMemoryError -XX:+HeapDumpOnOutOfMemoryError
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2022-12-01T17:39:06,229 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager - Initializing plugins manager...
2022-12-01T17:39:06,682 [INFO ] main org.pytorch.serve.ModelServer -
Torchserve version: 0.5.3
TS Home: /Users/XXX/opt/anaconda3/lib/python3.8/site-packages
Current directory: /Volumes/workplace/python_env/serve
Temp directory: /var/folders/w6/s5gp9htn2pb9z87lwp6fzjg9hv4nys/T/
Number of GPUs: 0
Number of CPUs: 12
Max heap size: 4096 M
Python executable: /Users/XXX/opt/anaconda3/bin/python
Config file: /Users/XXX/workplace/python_env/serve/config.properties
Inference address: http://0.0.0.0:8080
Management address: http://0.0.0.0:8081
Metrics address: http://127.0.0.1:8082
Model Store: /Volumes/workplace/python_env/serve/model_store
Initial Models: N/A
Log dir: /Volumes/workplace/python_env/serve/logs
Metrics dir: /Volumes/workplace/python_env/serve/logs
Netty threads: 32
Netty client threads: 0
Default workers per model: 2
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Limit Maximum Image Pixels: true
Prefer direct buffer: True
Allowed Urls: [file://.*|http(s)?://.*]
Custom python dependency for model allowed: true
Metrics report format: prometheus
Enable metrics API: true
Workflow Store: /Volumes/workplace/python_env/serve/model_store
Model config: N/A
2022-12-01T17:39:06,694 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager -  Loading snapshot serializer plugin...
2022-12-01T17:39:06,721 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: KQueueServerSocketChannel.
2022-12-01T17:39:06,795 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://0.0.0.0:8080
2022-12-01T17:39:06,795 [INFO ] main org.pytorch.serve.ModelServer - Initialize Management server with: KQueueServerSocketChannel.
2022-12-01T17:39:06,796 [INFO ] main org.pytorch.serve.ModelServer - Management API bind to: http://0.0.0.0:8081
2022-12-01T17:39:06,796 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: KQueueServerSocketChannel.
2022-12-01T17:39:06,797 [INFO ] main org.pytorch.serve.ModelServer - Metrics API bind to: http://127.0.0.1:8082
Model server started.

[david-waterworth]

Ahh I see, looks like for that to work you also need to use --ts-config config.properties

if you use torchserve --ncs --start it complains Missing mandatory parameter --model-store