Setting up Per User Secrets and other customizations with Jupyterhub on Kubernetes
While we’ve already set up Jupyterhub using zero-to-jupyterhub on Kubernetes, I wanted to expand access to my other prospective co-authors as part of trying to convince them to write with me (again) :). While I generally speaking trust my co-authors, I still like having some kind of access controls, so we don’t accidentally stomp on each-others work.
In Kubernetes the main way of doing this is with service accounts and secrets. While we can configure this globally, configuring this per-user needs a bit of custom code.
debug:
enabled: true
custom:
users:
holdenk:
secrets:
- env_name: MINIO_ACCESS_KEY
secret_name: minio
secret_key: accessKey
- env_name: MINIO_SECRET_KEY
secret_name: minio
secret_key: secretKey
service_account: holdenk
hub:
extraConfig:
preSpawnHook: |
import z2jh
async def my_pre_spawn_hook(spawner):
users = z2jh.get_config('custom.users') or {}
username = spawner.user.name
if username in users:
user = users[username]
print(user)
if 'service_account' in user:
spawner.service_account = user['service_account']
if 'secrets' in user:
secrets = user['secrets']
for secret in secrets:
name = secret['env_name']
spawner.env[name] = {
'valueFrom': {
'secretKeyRef': {
'name': secret['secret_name'],
'key': secret['secret_key']
}
}
}
c.KubeSpawner.pre_spawn_hook = my_pre_spawn_hook
config:
GitHubOAuthenticator:
client_id: YOURSECRET
client_secret: YOURSECRET
oauth_callback_url: YOURURL
allowed_organizations:
- scalingpythonml
scope:
- read:user
Authenticator:
admin_users:
- holdenk
JupyterHub:
authenticator_class: github
ingress:
enabled: true
annotations:
# With traefik 2+ this makes SSL only.
traefik.ingress.kubernetes.io/router.entrypoints: web, websecure
traefik.ingress.kubernetes.io/router.tls: "true"
# kubernetes.io/tls-acme: "true"
hosts:
- jupyter.pigscanfly.ca
tls:
- hosts:
- jupyter.pigscanfly.ca
secretName: k3s-jupyter-tls
singleuser:
memory:
limit: 10G
guarantee: 10G
cpu:
limit: 4
guarantee: 1
profileList:
- display_name: "Minimal environment"
description: "To avoid too much bells and whistles: Python."
default: true
- display_name: "Dask container"
description: "If you want to run dask"
kubespawner_override:
image: holdenk/dask-notebook:2020.1.1
- display_name: "Spark 3.1.1.11 container"
description: "If you want to run Spark"
kubespawner_override:
image: holdenk/spark-notebook:v3.1.1.11
- display_name: "Ray"
description: "If you want to run Ray"
kubespawner_override:
image: holdenk/ray-ray-nb:nightly
prePuller:
continuous:
# I've got a bunch of images that I use rarely
# in "real life" you probably want to leave this optimization on.
enabled: false
hook:
# I've got a bunch of images that I use rarely and
# in "real life" you probably want to leave this optimization on.
enabled: false
Most of the magic is inside preSpawnHook
. This also uses the z2jh
library to allow it to load the config in custom.users
in the YAML file. You could also point this to a database or something else, but given I’ve got about three users I figured in-line YAML was good enough for my case.
I’d like to thank consideRatio for all his help, to be clear any mistakes are my own fault.