Installation of Triton Server with helm chart

Description Hi,

I am trying to install the Triton Inference Server on Kubernates cluster. I am getting the following error:

helm install --generate-name .

Error: unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(Deployment.spec.template.spec.containers[0].securityContext): unknown field "fsGroup" in io.k8s.api.core.v1.SecurityContext

Triton Information Using master branch, I see the deployment.yaml is the same on most latest branches.

Expected behavior Deployment fails, expecting to create a deployment on Kubernetes.

Thank you

Svetlana

[xiaofei-du]

You don’t need to use the Triton metrics, or prometheus, if you don’t want to.

I managed to get the Triton Inference Server to run on Kubernetes 1.18. Maybe its worth updating the helm yaml files. I also added configuration in deployment.yaml to be able to use a local /model-repository, which is really useful if you don’t want to add your models to gs.

You don’t need to use the Triton metrics, or prometheus, if you don’t want to.

I managed to get the Triton Inference Server to run on Kubernetes 1.18. Maybe its worth updating the helm yaml files. I also added configuration in deployment.yaml to be able to use a local /model-repository, which is really useful if you don’t want to add your models to gs.

Hi, my k8s is 1.17 and i use a local model repository, but failed. So, can you tell me what did you do in detail? Thx

You can use hostPath to mount your local model repository into your pod:

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  ...
spec:
  ...
  template:
    ...
    spec:
      volumes:
      - name: triton-volume
        hostPath:
          # Directory location on host
          path: {{ .Values.image.modelRepositoryPath }}
          type: Directory
      containers:
          ...
          volumeMounts:
          - mountPath: /models
            name: triton-volume
            ...
          args: ["tritonserver", "--model-store=/models"]
          ...

Just be aware hostPath is not recommended to be used (Kubernetes Storage Lingo 101) since it breaks the k8s principle: workload portability.

[svetlana41]

You don’t need to use the Triton metrics, or prometheus, if you don’t want to.

I managed to get the Triton Inference Server to run on Kubernetes 1.18. Maybe its worth updating the helm yaml files. I also added configuration in deployment.yaml to be able to use a local /model-repository, which is really useful if you don’t want to add your models to gs.

Can you let me know the configuration in deployment.yaml to be able to use local model-repository?

Hi,

I have this in ‘values.yaml’ with the model repository path:

` replicaCount: 1

image: imageName: nvcr.io/nvidia/tritonserver:20.03-py3 pullPolicy: IfNotPresent modelRepositoryPath: /models numGpus: 1

service: type: LoadBalancer`

And this in deployment.yaml:

`apiVersion: apps/v1 kind: Deployment metadata: name: {{ template “triton-inference-server.fullname” . }} namespace: {{ .Release.Namespace }} labels: app: {{ template “triton-inference-server.name” . }} chart: {{ template “triton-inference-server.chart” . }} release: {{ .Release.Name }} heritage: {{ .Release.Service }} spec: replicas: {{ .Values.replicaCount }} selector: matchLabels: app: {{ template “triton-inference-server.name” . }} release: {{ .Release.Name }} template: metadata: labels: app: {{ template “triton-inference-server.name” . }} release: {{ .Release.Name }}

spec:
  containers:
    - name: {{ .Chart.Name }}
      image: "{{ .Values.image.imageName }}"
      imagePullPolicy: {{ .Values.image.pullPolicy }}

      resources:
        limits:
          nvidia.com/gpu: {{ .Values.image.numGpus }}

      args: ["trtserver", "--model-store={{ .Values.image.modelRepositoryPath }}"]

. . . . . volumes: - name: tritonvol hostPath: path: /home/to/model_repository `

This is a partial file.

Svetlana