[BUG] - error deploying QHub on AWS on 0.4

OS system and architecture in which you are running QHub

Ubuntu, x86_64

Expected behavior

QHub deploys a new cluster

Actual behavior

Qhub failed deployment at stage 3

How to Reproduce the problem?

Conda installed the latest qhub. Verified I got the latest release:

$ qhub --version
0.4.0

Initiated qhub with qhub init aws --project qhub-demo --ci-provider github-actions

Deployed with qhub deploy -c qhub-config.yaml

Failed the first time, and then I changed the project name in qhub-config.yaml to be qhubdemo supposing the extra dash was causing the error. No luck.

Below is my qhub config

project_name: qhubdemo
provider: aws
domain: qhubtest.cloud
certificate:
  type: self-signed
security:
  authentication:
    type: GitHub
    config:
      client_id: 5fefee...
      client_secret: 4qgf...
  keycloak:
    initial_root_password: FcpW...
default_images:
  jupyterhub: quansight/qhub-jupyterhub:v0.4.0
  jupyterlab: quansight/qhub-jupyterlab:v0.4.0
  dask_worker: quansight/qhub-dask-worker:v0.4.0
  dask_gateway: quansight/qhub-dask-gateway:v0.4.0
storage:
  conda_store: 60Gi
  shared_filesystem: 100Gi
theme:
  jupyterhub:
    hub_title: QHub - qhub-demo
    hub_subtitle: Autoscaling Compute Environment on Amazon Web Services
    welcome: Welcome to None. It is maintained by <a href="http://quansight.com">Quansight
      staff</a>. The hub's configuration is stored in a github repository based on
      <a href="https://github.com/Quansight/qhub/">https://github.com/Quansight/qhub/</a>.
      To provide feedback and report any technical problems, please use the <a href="https://github.com/Quansight/qhub/issues">github
      issue tracker</a>.
    logo: /hub/custom/images/jupyter_qhub_logo.svg
    primary_color: '#4f4173'
    secondary_color: '#957da6'
    accent_color: '#32C574'
    text_color: '#111111'
    h1_color: '#652e8e'
    h2_color: '#652e8e'
helm_extensions: []
monitoring:
  enabled: true
cdsdashboards:
  enabled: true
  cds_hide_user_named_servers: true
  cds_hide_user_dashboard_servers: false
ci_cd:
  type: github-actions
  branch: main
  commit_render: true
terraform_state:
  type: remote
namespace: dev
qhub_version: 0.4.0
amazon_web_services:
  region: us-west-2
  kubernetes_version: '1.22'
  node_groups:
    general:
      instance: m5.xlarge
      min_nodes: 1
      max_nodes: 1
    user:
      instance: m5.large
      min_nodes: 1
      max_nodes: 5
    worker:
      instance: m5.large
      min_nodes: 1
      max_nodes: 5
profiles:
  jupyterlab:
  - display_name: Small Instance
    description: Stable environment with 1 cpu / 4 GB ram
    default: true
    kubespawner_override:
      cpu_limit: 1
      cpu_guarantee: 0.75
      mem_limit: 4G
      mem_guarantee: 2.5G
  - display_name: Medium Instance
    description: Stable environment with 2 cpu / 8 GB ram
    kubespawner_override:
      cpu_limit: 2
      cpu_guarantee: 1.5
      mem_limit: 8G
      mem_guarantee: 5G
  dask_worker:
    Small Worker:
      worker_cores_limit: 1
      worker_cores: 0.75
      worker_memory_limit: 4G
      worker_memory: 2.5G
      worker_threads: 1
    Medium Worker:
      worker_cores_limit: 2
      worker_cores: 1.5
      worker_memory_limit: 8G
      worker_memory: 5G
      worker_threads: 2
environments:
  environment-dask.yaml:
    name: dask
    channels:
    - conda-forge
    dependencies:
    - python
    - ipykernel
    - ipywidgets
    - qhub-dask ==0.3.13
    - python-graphviz
    - numpy
    - numba
    - pandas
  environment-dashboard.yaml:
    name: dashboard
    channels:
    - conda-forge
    dependencies:
    - python==3.9.7
    - ipykernel==6.4.1
    - ipywidgets==7.6.5
    - qhub-dask==0.3.13
    - param==1.11.1
    - python-graphviz==0.17
    - matplotlib==3.4.3
    - panel==0.12.4
    - voila==0.2.16
    - streamlit==1.0.0
    - dash==2.0.0
    - cdsdashboards-singleuser==0.6.0

Command output

INFO:qhub.provider.terraform:terraform init took 6.004 [s]
INFO:qhub.provider.terraform:terraform apply directory=stages/03-kubernetes-initialize targets=[]
INFO:qhub.provider.terraform: terraform at /tmp/terraform/1.0.5/terraform
[terraform]: Acquiring state lock. This may take a few moments...
[terraform]: ╷
[terraform]: │ Error: error reading EKS Cluster (qhubdemo-dev): couldn't find resource
[terraform]: │ 
[terraform]: │   with data.aws_eks_cluster.default,
[terraform]: │   on _qhub.tf.json line 28, in data.aws_eks_cluster.default:
[terraform]: │   28:             }
[terraform]: │ 
[terraform]: ╵
[terraform]: Releasing state lock. This may take a few moments...

Problem encountered: Terraform error

Versions and dependencies used.

$ conda --version
conda 4.11.0

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.0", GitCommit:"cb303e613a121a29364f75cc67d3d580833a7479", GitTreeState:"clean", BuildDate:"2021-04-08T16:31:21Z", GoVersion:"go1.16.1", Compiler:"gc", Platform:"linux/amd64"}
error: You must be logged in to the server (the server has asked for the client to provide credentials)

$ qhub --version
0.4.0

[tylerpotts]

@iameskild Hey man!

You the best! That fixed it. I created a PR in #1234 to prevent this from happening to others

[viniciusdc]

@viniciusdc I don’t think so. It’s a pretty vanilla AWS account. I did go into the console and confirm there was an EKS cluster with the expected name in the correct region

I see, could you enable TF_LOG=debug and rerun the deployment? this might give us more info as to what resource name terraform itself is looking for