question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

How to Use TLS in Dask-Gateway?

See original GitHub issue

I understand that it is recommended to use TLS in a production environment as per the docs, so I’m trying to set that up. Here are the steps I followed for my attempt at doing this:

[1] I added the paths to the self-signed certificate and key files in a Dockerfile [2] Pushed that Docker Image to Google Cloud Image Repository [3] Replace image names from “daskgateway/dask-gateway-server” to the image in Google Cloud Image Repo in the helm config file [4] Added the paths to the self-signed key and cert files in the extraConfig field

What happened: Nothing changed. No errors. The internal load balancer for the traefik proxy server still using HTTP.

What you expected to happen: I expected the internal load balancer to use HTTPS

Minimal Complete Verifiable Example:

Dockerfile:

FROM daskgateway/dask-gateway-server:latest

ADD certs /certs/

Helm Config:

extraConfig:
    security: |
      c.Proxy.tls_cert = "certs/myca.pem"
      c.Proxy.tls_key = "certs/mykey.pem"
    clusteroptions: |
      from dask_gateway_server.options import Options, Integer, Float, String

      c.KubeClusterConfig.idle_timeout = 3600

      def option_handler(options):
        return {
        "worker_cores": options.worker_cores,
        "worker_memory": "%fG" % options.worker_memory,
        "image": options.image,
        }

      c.Backend.cluster_options = Options(
        Integer("worker_cores", 2, min=1, max=8, label="Worker Cores"),
        Float("worker_memory", 4, min=1, max=16, label="Worker Memory (GiB)"),
        String("image", default="daskgateway/dask-gateway:latest", label="Image"),
        handler=option_handler,
      )

Environment:

  • Helm version: 0.8.0

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:8 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
cdibblecommented, May 28, 2021

Thanks for the tips! I’m still pretty new to kubernetes and looking into the DNS names has been edifying.

I wanted to post this snippet as reference for others. I was able to hide Dask Gateway, including the Dask Dashboards, behind my VPN using the following annotations on the traefik service. Note that these annotations are specific to AWS EKS with AWS Elastic Load Balancers (and AWS Load Balancer Controller as the Ingress Controller), but I’d think there are similar methods with other load balancers.

traefik:
  service:
    type: LoadBalancer # Use LoadBalancer if you want internet-facing ingress.
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-type: alb
      service.beta.kubernetes.io/aws-load-balancer-internal: <CIDR-block-for-local-VPC-traffic>

[edit- fixed indentations]

0reactions
droctothorpecommented, May 27, 2021

The other services will be exposed (via the same ELB that serves the dashboard). The advantage of using the K8s DNS names is that requests from the in-cluster Dask Gateway clients to the Gateway API don’t leave the cluster for no reason. It also helps with environment file consistency as well if you’re provisioning to multiple discrete environments. It’s a nice to have but not strictly necessary and kind of a tangent from your original question, heh.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Security settings — Dask Gateway 2022.11.0 documentation
If you have your own TLS certificate/key pair, you can specify the file locations in your dask_gateway_config.py file. The relevant configuration fields are:...
Read more >
Multiple processes per worker while using gateway - Distributed
Hi I'm having a hard time figuring out why when I configure my dask clusters (created via gateway) to have multiple processes (instead...
Read more >
Dask: How to Add Security (TLS/SSL) to Dask Cluster?
I'm trying to figure out how to add a security layer to my Dask Cluster deployed using helm on GKE ...
Read more >
Dask-Gateway Internals - Jim Crist-Harif
... Dask-Gateway - **Centrally Managed** - Admins do the heavy lifting, users get easy cluster access. - **Secure by default** - Automatic TLS...
Read more >
Deployment: Dask - Prefect Docs
In our case, we want to use Prefect's DaskExecutor to submit task runs to a ... Dask cluster via Dask Gateway you may...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found