Moving users/groups definition from qhub-config.yaml to Keycloak

Background

Keycloak has been added to Qhub in development main branch. This allows users and groups to be created and managed through the Keycloak web console. It is also possible to configure Keycloak so that, for example, users are federated from an external database such as LDAP.

In previous versions (and current, v0.3.13) of QHub, users and groups had to be added within the qhub-config.yaml file, and a redeployment of the whole system needed to be triggered.

In tandem with the move to Keycloak, uid and gid values are no longer required (in the YAML or anywhere else). Everyone is user jovyan and shared group folders are only mounted for users within those groups - previously, all shared group folders were visible but were only accessible to authorized users.

A separate ‘root’ user has a password into the Keycloak web console. That user can manage Keycloak but is not a regular QHub user - it cannot login to the usual QHub JupyterHub/Lab area etc.

Discussion

As part of the upgrade to a Keycloak-enabled version of QHub, a new qhub upgrade command will process an existing qhub-config.yaml. It will remove the users and groups keys from the YAML file permanently, and will also spit out a JSON file containing the users and groups. That file is intended to be used as a one-off import into the new Keycloak instance once it is running. It can be imported through the Keycloak web console.

Going forward, users and groups will be managed through the Keycloak web console. I believe this is more natural for most admins, and avoids the need for redeployment. However, I am aware that heavy ‘gitops’ users may miss the ability to edit the YAML file to add/remove users.

Right now in main branch, it is still possible to add users and groups keys to the qhub-config.yaml file (uid/gid no longer allowed or needed). These users/groups will be synced with the Keycloak database when redeployed. However, these then become managed by Terraform meaning that the users/groups will be deleted automatically if they are dropped from the YAML file at some point (although it is technically possible to remove them from Terraform management). If users/groups are deleted from Keycloak manually, they will be reinstated whenever redeployment happens. However, that will not be the case when manually-added users are manually deleted. If a user is added manually to Keycloak, an error will occur if that same user is subsequently added to the qhub-config.yaml file.

While it could be tempting to allow both approaches - YAML or Keycloak web console - to be available alongside each other, I believe it will be confusing and lead to mistakes in user management.

My proposal is to remove this temporary functionality (users/groups still being allowed in qhub-config.yaml) from the main branch before the next full release of QHub.

Please let us know if you have any views about the proposed move to managing users/groups in Keycloak, or about making user/group management unavailable in the YAML file.

Value/benefit

This will leave only one approach to user management that we need to support, in a configurable environment (Keycloak), and will cause less confusion for admins. It avoids having an extra choice that admins need to make at deployment time.

[danlester]

I will move forward with a PR for my proposal as described initially.

[rsignell-usgs]

I like the suggested approach of having only one way to manage users, and I’m in favor of only one way to manage the users. While I did like just editing the yaml file to add new users, we now have 90 users and it’s becoming a bit annoying that most of the config is just user info.

We already are in the position with the ESIP Qhub that we need to deploy a new version and then migrate user data, so we are fine doing this all at once when v0.4 is released.