Use kubernetes secrets for passing sensitive environment variables to pods

Update by @consideRatio

The environment variable should in order to follow common k8s security practices be mounted from a k8s Secret rather than set directly as a name/value pair in the k8s Pod resource manifest.

# like this...
spec:
  containers:
  - name: users-jupyter-server-container
    env:
      - name: JUPYTERHUB_API_TOKEN
        valueFrom:
          secretKeyRef:
            name: user1-jupyterhub-api-token
            key: JUPYTERHUB_API_TOKEN

# instead of liket his
spec:
  containers:
  - name: users-jupyter-server-container
    env:
      - name: JUPYTERHUB_API_TOKEN
        value: secret-stuff

The difference is that if you do kubectl get pod -o yaml jupyter-user1 then you will see the secret-stuff value if we mount it like we do now, but not if we reference a k8s Secret in order to mount the environment variable. But, for a user with ability to do kubectl get secret -o yaml user1-jupyterhub-api-token then it doesn’t matter I think.

Original issue text

https://github.com/jupyterhub/kubespawner/blob/892780ae86c6e74172065ab984bf2d059d00c31f/kubespawner/spawner.py#L1498

https://github.com/jupyterhub/jupyterhub/blob/914a3eaba5c83f639b4b5d8873c6e9d93e809f31/jupyterhub/spawner.py#L722

[minrk]

I think tackling this will give us a chance to generally improve logic around “objects created to go with the pod”. Using consistent labels for any object created by kubespawner associated with a user pod should make it manageable to create and clean up everything without lifecycle concerns when e.g. gc is unavailable due to ordering.

[yuvipanda]

This makes sense. The complexity here will be creating & cleaning up the secret objects with pod start / stop. We could explore doing this with kubernetes garbage collection, although since the secret would need to be created before the pod - not sure if that’ll work for that.

We already create PVCs before creating pods, I think we can extend that logic to secrets. While we’re there, we could possibly just put all env vars there instead of just ‘sensitive’ ones.

Happy to review any PRs 😃