question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] Unauthorized client error in flyteadmin - K8S with Keycloak

See original GitHub issue

Describe the bug

I am using Keycloak as identity provider in my setup.

I have setup 2 clients, flytectl(access type public) and flytepropeller(access type confidential with client Id and secret).

I am following the keycloak section in the https://docs.flyte.org/en/latest/deployment/cluster_config/auth_setup.html#

My relevant values in the helm values file are as below

secrets: adminOauthClientCredentials: # – If enabled is true, helm will create and manage flyte-secret-auth and populate it with clientSecret. # If enabled is false, it’s up to the user to create flyte-secret-auth as described in # https://docs.flyte.org/en/latest/deployment/cluster_config/auth_setup.html#oauth2-authorization-server enabled: true clientSecret: “CO2n3hovN0J78FqxxOVtjOtkHH5fPL9C” clientId: “flytepropeller”

– FlyteAdmin server configuration

adminServer: # Refer to the server config. server: httpPort: 8088 grpcPort: 8089 security: # – Controls whether to serve requests over SSL/TLS. secure: false # – Controls whether to enforce authentication. Follow the guide in https://docs.flyte.org/ on how to setup authentication. useAuth: true allowCors: true allowedOrigins: # Accepting all domains for Sandbox installation - “*” allowedHeaders: - “Content-Type” - “flyte-authorization” # Refer to the full structure for documentation. flyteadmin: roleNameKey: “iam.amazonaws.com/role” profilerPort: 10254 metricsScope: “flyte:” metadataStoragePrefix: - “metadata” - “admin” eventVersion: 2 testing: host: http://flyteadmin # – Authentication configuration auth: authorizedUris: # This should point at your public http Uri. - https://flytedeployment url # This will be used by internal services in the same namespace as flyteadmin - http://flyteadmin:80 # This will be used by internal services in the same cluster but different namespaces - http://flyteadmin.nmlp.svc.cluster.local:80

  # Controls app authentication config
  appAuth:
    thirdPartyConfig:
      flyteClient:
        clientId: flytectl
        redirectUri: https://<flyte deployment url>/callback
        scopes:
          - offline
          - all

  # Controls user authentication
  userAuth:
    openId:
      baseUrl: https://<keycloak production realm>/realms/nsdmlp
      scopes:
        - profile
        - openid
      clientId: flytepropeller

I have also edited the flyte-admin-secrets to have the correct client secret

apiVersion: v1 data: claim_symmetric_key: cWlBYzlYWHdLN3lnaksrWUJGdStFUlRYK0RDdlk4SjVjZFJtaXBTcDBhdw== cookie_block_key: ejZPdkhrZ1crWXdib21JZHdVZ05IOGJESVp0OE5KWnNZT285KzIyRVM1dw== cookie_hash_key: Q093TUY2RTdOMW5MeFZ4Rnk1dGNzZGN5NU5aeTNWN2JTMXRPTjBLUGhQQ2JWZ3hGby9XQkVRdi84Yjk4ZEIyeEV3Zm5KYURDVzFkSjBuSGZrbS8zYVE= oidc_client_secret: CO2n3hovN0J78FqxxOVtjOtkHH5fPL9C

With all the above setting, when i try to login to the flyte console via the ingress, it gives me the below message in the flyteadmin pod logs

{“json”:{},“level”:“error”,“msg”:“Failed to retrieve tokens from request, redirecting to login handler. Error: [EMPTY_OAUTH_TOKEN] Failure to retrieve cookie [flyte_idt], caused by: http: named cookie not present”,“ts”:“2022-08-04T18:34:11Z”} {“json”:{},“level”:“error”,“msg”:“Error when exchanging code oauth2: cannot fetch token: 401 Unauthorized\nResponse: {"error":"unauthorized_client","error_description":"Invalid client secret"}”,“ts”:“2022-08-04T18:34:12Z”} Logs from 8/5/2022, 12:02:39 AM

And the chrome errors out with 403 access denied page.

I have tried secret and id with and without quotes(just in case if helm was acting crazy) but it still gives the same error.

Please assist.

Thanks, Sujith

Expected behavior

Setting up the secret and client must enable the correct oauth flow and let me login to the flyte

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn’t been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:6 (5 by maintainers)

github_iconTop GitHub Comments

2reactions
samuel-sujithcommented, Aug 5, 2022

it was a basic mistake from my side.

Instead of putting the oidc_client_secret in the stringdata section, I had put in the data section, which was causing the secret to be not fetched correctly.

This is working fine now.

Thanks a lot for all the slack assistance.

1reaction
samuel-sujithcommented, Aug 5, 2022

The propeller error went away after I set the NO_PROXY env in propeller pod to flyteadmin. But the flyteadmin error is still there

{“json”:{},“level”:“error”,“msg”:“Error when exchanging code oauth2: cannot fetch token: 401 Unauthorized\nResponse: {"error":"unauthorized_client","error_description":"Invalid client secret"}”,“ts”:“2022-08-05T04:18:48Z”}

Read more comments on GitHub >

github_iconTop Results From Across the Web

[BUG] Unauthorized client error in flyteadmin - bytemeta
Describe the bug. I am using Keycloak as identity provider in my setup. I have setup 2 clients, flytectl(access type public) and flytepropeller(access...
Read more >
error: You must be logged in to the server (Unauthorized ...
I applied below to kubernetes. kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: keycloak-admin-group roleRef: apiGroup: ...
Read more >
Kubernetes Login Fail with Keycloak | by Scott Chang - Medium
error: You must be logged in to the server (Unauthorized). I got above error message when trying to login Kubernetes via Keycloak.
Read more >
Keycloak API returns 401 unauthorized sometimes
Just setting 2 replicas doesn't mean that Keycloak runs in a cluster, it's just a K8s cluster, but not a Keycloak cluster.
Read more >
Authentication in Flyte
Flyte ships with a canonical implementation of OpenIDConnect client and OAuth2 Server, integrating seamlessly into an organization's existing identity provider.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found