Installing BinderHub to existing JupyterHub

I am trying to install Binderhub on an existing Jupyterhub installation in a EKS cluster which has been running for a while.

I cannot make the Binderhub installation work. I have removed the dependency from the Binder Helm chart: binderhub/helm-chart/binderhub/Chart.yaml at 0fa35a06e7f7d054f8bb8dd2cba158ed7f45ac68 · jupyterhub/binderhub · GitHub

Also, I merged the bundle Binderhub into the Jupyterhub config:

But it still not working. Is this is even possible?

*** Note, my current Jupyterhub installation is using Okta verification

It’s definitely possible, though to give you more help you’ll need to tell us what isn’t working! As well as describing exactly what you’re seeing it would help if shared your logs, and your full configuration.

Thanks for your response. The biggest challenge I’ve found is how to integrate the exiting Okta verification from Jupuyterhub.
This is my configuration for Binder values file.

config:
  BinderHub:
    auth_enabled: true
    base_url: /
    build_node_selector: {}
    hub_url: https://jupyterhub.example.com
    use_registry: true
    image_prefix: binder/test-binder-
  KubernetesBuildExecutor: {}
  GitHubRepoProvider:
    access_token: <Redacted>
    
jupyterhub:
  cull:
    users: false
  hub:
    allowNamedServers: true
    namedServerLimitPerUser: 5
    config:
      JupyterHub:
        authenticator_class: generic-oauth
        redirect_to_server: false
      BinderSpawner:
        auth_enabled: true
    loadRoles:
      user:
        scopes:
          - self
          - "access:services!service=binder"

  singleuser:
    cmd: jupyterhub-singleuser

ingress:
  enabled: true
  https:
    enabled: false
    type: kube-lego
  hosts: []
  ingressClassName:
  annotations: []

And this for Jupyterhub values

hub:
  service:
      binder:
        apiToken:  <Redacted>

  loadRoles:
    user:
      scopes:
        - self
        - "access:services!service=binder"

And merged the bundle spawner into the jupyterhub.

I am seeing this error in the Binder pod:

[W 240628 11:30:24 auth:1128] Detected unused OAuth state cookies

The jupyterhub section of your Binder values has no effect if you’re using an existing JupyterHub, everything needs to go into your JupyterHub values.

It looks like there’s no way to disable the jupyterhub bundled in the BinderHub helm chart so for now you’ll have a second useless hub deployment, I’ll open an issue for that. It shouldn’t affect your deployment though.

Actually I was wrong, it looks like the BinderHub chart makes several assumptions about the presence of the bundled JupyterHub. This is mostly to make it easier to configure a new deployment of BinderHub. It looks like it’s non-trivial to separate them:

Thanks @manics.
It totally makes sense. I have literally tried everything but I cant make it work.

I will try few more things and keep this topic up to date.

The limitation is due to the design of the BinderHub Helm chart, but it’s not an inherent limitation of the BinderHub application.

For example

shows how to run BinderHub as a managed JupyterHub service:

This is effectively the opposite of the dependency in the Helm chart.

Thanks @manics .
After checking the discussions I think I will try with Binderhub Service
Do you have any experience with it by any chance?

1 Like

I’ve never used it, though I’ll take a look when I get time.

@manics FYI, I was able to install Binderhub successfully in my existing Jupyterhub using the Binderhub helm chart.
Here are my configuration parameters:

  • Binder values file
pdb:
  enabled: true
  maxUnavailable: 1
  minAvailable:

replicas: 1

resources:
  requests:
    cpu: 0.2
    memory: 512Mi

rbac:
  enabled: true

nodeSelector: {}
tolerations: []

image:
  name: quay.io/jupyterhub/k8s-binderhub
  tag: "1.0.0-0.dev.git.3468.h30c3bc9"
  pullPolicy: ""
  pullSecrets: []

# registry here is only used to create docker config.json
registry:
  username: <Redacted>
  password: 

service:
  type: ClusterIP
  labels: {}
  annotations:
    prometheus.io/scrape: "true"
  nodePort:
  loadBalancerIP:

config:
  GitHubRepoProvider:
    access_token: <Redacted>
  # These c.BinderHub properties are referenced by the Helm chart
  BinderHub:
    base_url: /
    build_node_selector: {}
    hub_url: <Redacted>
    hub_url_local: http://hub:8081
    use_registry: true
    image_prefix: <Redacted>/binder-
    auth_enabled: true
  KubernetesBuildExecutor: {}

extraConfig: {}

extraFiles: {}

extraPodSpec: {}

deployment:
  readinessProbe:
    enabled: true
    initialDelaySeconds: 0
    periodSeconds: 5
    failureThreshold: 1000 # we rely on the liveness probe to resolve issues if needed
    timeoutSeconds: 3
  livenessProbe:
    enabled: true
    initialDelaySeconds: 10
    periodSeconds: 5
    failureThreshold: 3
    timeoutSeconds: 10
  labels: {}

imageBuilderType: "dind"

dind:
  initContainers: []
  daemonset:
    image:
      name: docker.io/library/docker
      tag: "26.1.3-dind" # ref: https://hub.docker.com/_/docker/tags
      pullPolicy: ""
      pullSecrets: []
    # Additional command line arguments to pass to dockerd
    extraArgs: []
    lifecycle: {}
    extraVolumes: []
    extraVolumeMounts: []
  storageDriver: overlay2
  resources: {}
  hostSocketDir: /var/run/dind
  hostLibDir: /var/lib/dind
  hostSocketName: docker.sock

# Podman in Kubernetes
pink:
  initContainers: []
  daemonset:
    image:
      name: quay.io/podman/stable
      tag: "v5.0.3" # ref: https://quay.io/repository/podman/stable
      pullPolicy: ""
      pullSecrets: []
    lifecycle: {}
    extraVolumes: []
    extraVolumeMounts: []
  resources: {}
  hostStorageDir: /var/lib/pink/storage
  hostSocketDir: /var/run/pink
  hostSocketName: podman.sock

imageCleaner:
  enabled: true
  image:
    name: quay.io/jupyterhub/docker-image-cleaner
    tag: "1.0.0-beta.3"
    pullPolicy: ""
    pullSecrets: []
  # delete an image at most every 5 seconds
  delay: 5
  # Interpret threshold values as percentage or bytes
  imageGCThresholdType: "relative"
  # when 80% of inodes are used,
  # cull images until it drops below 60%
  imageGCThresholdHigh: 80
  imageGCThresholdLow: 60
  # cull images on the host docker as well as dind
  # configuration to use if `imageBuilderType: host` is configured
  host:
    dockerSocket: /var/run/docker.sock
    dockerLibDir: /var/lib/docker

ingress:
  annotations:
    kubernetes.io/ingress.class: traefik
  enabled: true
  hosts: []
  pathSuffix: ""
  tls: []

initContainers: []
lifecycle: {}
extraVolumes: []
extraVolumeMounts: []
extraEnv: {}
podAnnotations: {}

# Deprecated values, kept here so we can provide useful error messages
cors: {}

global: {}

I also modified the environment variables from the Binder deployment file:

        - name: JUPYTERHUB_API_TOKEN
          value: <Redacted>
        - name: JUPYTERHUB_SERVICE_NAME
          value: binder
        - name: JUPYTERHUB_API_URL
          value: "<jupytehub url>/hub/api/"
        - name: JUPYTERHUB_BASE_URL
          value: /
        - name: JUPYTERHUB_CLIENT_ID
          value: "service-binderhub"
        - name: JUPYTERHUB_OAUTH_CALLBACK_URL
          value: <binderhub url>/oauth_callback"

Finally, I added the Binder service in my Jupyterhub helm chart values:

hub:
  services:
    binder:
      apiToken: <Redacted>
      admin: true
      oauth_client_id: service-binderhub
      oauth_no_confirm: true
      oauth_redirect_uri: "<binderhub url>/oauth_callback"

However, I am facing a last challenge. When launching my server in Binderhub it is not pulling the Binder image but the image from my Jupyterhub profile. @manics Do you know how can I ‘mix’ the Binder Spawners with the ones I already have for Jupyterhub?

You’ll need to get BinderSpawnerMixin into your JupyterHub container, for example by copying and pasting it into your Z2JH extraConfig:

and configure your auth settings in that spawner (this is an example of using DockerSpawner locally instead of KubeSpawner, but the config should be similar):

though you could hard code this since you know you’re using auth.

I have implemented those changes but I have started to get Spawn failed in Binderhub.
The pod tries to initialize it gets stuck. These are the logs:

/srv/conda/envs/notebook/lib/python3.10/subprocess.py:961: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdout = io.open(c2pread, 'rb', bufsize)
[I 03:56:02.373 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret

  _   _          _      _
 | | | |_ __  __| |__ _| |_ ___
 | |_| | '_ \/ _` / _` |  _/ -_)
  \___/| .__/\__,_\__,_|\__\___|
       |_|

Read the migration plan to Notebook 7 to learn about the new features and the actions to take if you are using extensions.

https://jupyter-notebook.readthedocs.io/en/latest/migrate_to_notebook7.html

Please note that updating to Notebook 7 might break some of your extensions.

[W 03:56:03.063 NotebookApp] Loading JupyterLab as a classic notebook (v6) extension.
[W 2024-07-14 03:56:03.065 LabApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2024-07-14 03:56:03.065 LabApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2024-07-14 03:56:03.065 LabApp] 'base_url' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2024-07-14 03:56:03.065 LabApp] 'token' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2024-07-14 03:56:03.066 LabApp] 'trust_xheaders' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2024-07-14 03:56:03.066 LabApp] 'trust_xheaders' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2024-07-14 03:56:03.066 LabApp] 'trust_xheaders' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[I 2024-07-14 03:56:03.070 LabApp] JupyterLab extension loaded from /srv/conda/envs/notebook/lib/python3.10/site-packages/jupyterlab
[I 2024-07-14 03:56:03.070 LabApp] JupyterLab application directory is /srv/conda/envs/notebook/share/jupyter/lab
[C 03:56:03.075 NotebookApp] Running as root is not recommended. Use --allow-root to bypass.

I am not able to see any error, not sure why is not able to initialize.

Can you try setting c.BinderSpawner.cmd = "jupyterhub-singleuser" (or whatever you’ve called BinderSpawner)?

Thanks @manics. Merging the Binder Spawner and adding jupyterhub-singleuser worked.
Additionally, I had to include --allow-root setting.

1 Like