Support for SwiftStack S3 API

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I’m always frustrated when […]

I’m unable to connect to a repository located in SwiftStack using the S3 API. When attempting to start Triton, I get this error:

I1218 17:41:51.583433 1 server.cc:277] No server context available. Exiting immediately.
error: creating server: Internal - Could not get MetaData for bucket with name triton-repo

I’m able to connect to AWS s3 from Triton and load a model.  I’m also able to connect to SwiftStack using the AWS cli like this:

docker run -e "AWS_DEFAULT_REGION=z1-a" \
-e "AWS_ACCESS_KEY_ID=<key id>" \
-e "AWS_SECRET_ACCESS_KEY=<secret>" \
-it amazon/aws-cli --endpoint=https://z1-a.objectstorage.<domainname> s3 ls

Looking through the triton server code, it looks like it’s using the official AWS s3 client library, so I’d imagine the environment variables should be used in the same way, but maybe there’s a difference.  Here’s how I’m testing with triton:

docker run -e "AWS_DEFAULT_REGION=z1-a" \
-e “AWS_ACCESS_KEY_ID=<key id>" \
-e “AWS_SECRET_ACCESS_KEY=<secret>" \
--rm -p8000:8000 -p8001:8001 -p8002:8002 nvcr.io/nvidia/tritonserver:20.11-py3 tritonserver \
--model-repository=s3://z1-a.objectstorage.<domainname>:443/triton-repo/triton1 \
--strict-model-config=false --log-verbose=1

Describe the solution you’d like A clear and concise description of what you want to happen.

I would like to be able to load models from a repository located in SwiftStack using the S3 API. 😃

Describe alternatives you’ve considered A clear and concise description of any alternative solutions or features you’ve considered.

As a workaround I’m currently loading models from AWS, which works but is slower.

Additional context Add any other context or screenshots about the feature request here.

[arunraman]

I had no problem running this

docker run -e AWS_ACCESS_KEY_ID=<AWS_KEY_ID> -e AWS_SECRET_ACCESS_KEY=<AWS_SECRET_KEY> AWS_DEFAULT_REGION=<Region> --gpus device=1 --rm -p8000:8000 -p8001:8001 -p8002:8002 --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/tritonserver:20.12-py3 tritonserver --model-repository=s3://model_bucket/ --model-control-mode=poll --repository-poll-secs=5 --log-verbose=true --exit-on-error=false --strict-model-config=false

Are the double quotes on the environment variable an issue?

[CoderHam]

@barelyreal the issue will be fixed by https://github.com/triton-inference-server/server/pull/2879 and https://github.com/triton-inference-server/server/pull/2880