Customizing JupyterHub on Kubernetes

Hello,

sorry for such a newbie question, can you please advise how to customize JupyterHub Login page when using Kubernetes.

It is easy to edit login.html directly in JupyterHub installation, however when using Kubernetes (and helm chart) I am not sure how to do this?

What is the right approach from your point of view?

Thank you very much for any idea or advice that would help move into right direction.

Regards,
Peter

2 Likes

I’m having trouble with this too. Have you had any luck?

Hi,

You have to put the path, where your custom template takes place, into template_paths config of jupyterhub (see custom templates documentation):

jupyterhub:
  hub:
    extraConfig:
      templates: |
        c.JupyterHub.template_paths = ['/path/to/custom/templates']

And this path and templates must exist in jupyterhub container. There are 2 ways (that I am aware of) to do this. First you can extend the jupyterhub image by copying the templates and use this image in your config. But this means that you have to create a new image everytime you upgrade your jupyterhub. Second way, which I prefer, you can use initContainers, by which you can clone/download your custom templates into a volume and then you can mount this volume into the hub container. Here is an example config for that:

jupyterhub:
  hub:
    # clone custom JupyterHub templates into a volume
    initContainers:
      - name: git-clone-templates
        image: alpine/git
        args:
          - clone
          - --single-branch
          - --branch=master
          - --depth=1
          - --
          - https://github.com/your/repo.git
          - /etc/jupyterhub/custom
        securityContext:
          runAsUser: 0
        volumeMounts:
          - name: custom-templates
            mountPath: /etc/jupyterhub/custom
    extraVolumes:
      - name: custom-templates
        emptyDir: {}
    extraVolumeMounts:
      - name: custom-templates
        mountPath: /etc/jupyterhub/custom

    extraConfig:
      templates: |
        c.JupyterHub.template_paths = ['/etc/jupyterhub/custom/jupyterhub/templates']

This config clones the repo (https://github.com/your/repo.git, you have to change this with the url of repo where your custom templates are) into /etc/jupyterhub/custom folder, to where the custom-templates volume is mounted. The same volume will be mounted into hub container too (see extraVolumeMounts config). Finally we have to tell jupyterhub where to find custom templates, for that we have to set c.JupyterHub.template_paths as mentioned before. For example if your templates exist in jupyterhub/templates folder in your repo, then set it to ['/etc/jupyterhub/custom/jupyterhub/templates'] as I did in the example config.

I hope this helps :slight_smile:

3 Likes

:heart: @bitnik for your example!

Another perhaps more complicated option is to mount configmaps with the template files, the benefit is that you don’t need to have an init container do stuff but instead mounts things directly assuming you have these files available on helm upgrade etc and some additional configuration work.

To use this approach, do something like below. Note that this is a WIKI post allowing you to edit it if you find something to correct or add, please feel free to do so!

  1. use a custom helm chart that requires the jupyterhub helm chart

    Here is a requirements.yaml file as an example.

    # requirements.yaml
    dependencies:
      # CHART_VERSION: https://jupyterhub.github.io/helm-chart/
      - name: jupyterhub
        version: 0.9-dcde99a
        repository: https://jupyterhub.github.io/helm-chart/
    
  2. add some template files in a local folder alongside helm chart configuration files

    spawn.html - In the helm chart that has a requirements.yaml declaring jupyterhub as a dependency, this file could for example be added in files/etc/jupyterhub/templates/. Note that it references an image in /hub/static/external/my-custom-image.svg, this needs also to be mounted for use and that is done if you place it within files/static/external/ assuming the use of the configmaps presented in this example.

    {% extends "page.html" %}
    {% if announcement_spawn %}
    {% set announcement = announcement_spawn %}
    {% endif %}
    
    {% block main %}
    
    <div class="container">
      {% block heading %}
      <div class="row text-center">
    
        <img src="/hub/static/external/my-custom-image.svg" height="80px" />
    
        <h1>Server Options</h1>
      </div>
      {% endblock %}
      <div class="row col-sm-offset-2 col-sm-8">
        {% if for_user and user.name != for_user.name -%}
        <p>Spawning server for {{ for_user.name }}</p>
        {% endif -%}
        {% if error_message -%}
        <p class="spawn-error-msg text-danger">
          Error: {{error_message}}
        </p>
        {% endif %}
        <form enctype="multipart/form-data" id="spawn_form" action="{{url}}" method="post" role="form">
          {{spawner_options_form | safe}}
          <br>
          <input type="submit" value="Start" class="btn btn-jupyter form-control">
        </form>
      </div>
    </div>
    
    {% endblock %}
    
  3. make a configmap helm template add these files to itself

    # configmap.yaml that I install into the jupyterhub namespace
    # through a custom Helm chart that has a requirements.yaml file
    # that in turn installs jupyterhub.
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: hub-templates
    data:
      {{- (.Files.Glob "files/etc/jupyterhub/templates/*").AsConfig | nindent 2 }}
    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: hub-external
    binaryData:
      {{- $root := . }}
      {{- range $path, $bytes := .Files.Glob "files/static/external/*" }}
      {{ base $path }}: '{{ $root.Files.Get $path | b64enc }}'
      {{- end }}
    
  4. and then mount that configmap to the hub pod

    jupyterhub:
      hub:
        extraVolumes:
          - name: hub-templates
            configMap:
              name: hub-templates
          - name: hub-external
            configMap:
              name: hub-external
        extraVolumeMounts:
          - name: hub-templates
            mountPath: /etc/jupyterhub/templates
          - name: hub-external
            mountPath: /usr/local/share/jupyterhub/static/external
        extraConfig:
          templates: |
            c.JupyterHub.template_paths.insert(0, "/etc/jupyterhub/templates")
    
4 Likes

Thank you @consideRatio and @bitnik for your detailed examples and explanations! I will try them out.

Could I ask how you knew what values to put under extraVolumes or extraVolumeMounts? I looked over values.yaml but didn’t know what parameters to put (e.g.name and configMap under extraVolumes).

Another quick question:

extraConfig:
      templates: |
        c.JupyterHub.template_paths = ['/etc/jupyterhub/custom/jupyterhub/templates']

I’ve seen in the documentation where templates is replaced with other keywords, such as myConfig.py. Does the keyword chosen matter?

I had to depended on previous knowledge about Kubernetes, Helm, and inspiration from mybinder.org’s configuration. It took quite a while to get this right for me.

Reference reading:

One challenge that I struggled with was referencing images from my template:

<img src="/hub/static/external/my-custom-image.svg" height="80px" />

The question I asked myself was what content would the webserver that JupyterHub use try to serve if we wrote that? I figured out by some testing and source code inspection, that something within a HTML reference to /hub will map to content on a hard drive where JupyterHub’s data_files_path is configured to point to. I did not reconfigure this location but instead used the default value, which was /usr/local/share/jupyterhub/, and placed a folder of stuff within it using a ConfigMap.


I’m not 100% that you mean this, but…

extraConfig:
  whateverYouWantItDoesntMatter: |
    c.JupyterHub.template_paths = ['/etc/jupyterhub/custom/jupyterhub/templates']

Making extraConfig a dictionary of key value pairs allows extraConfig information from one file to be merged with another better. This could be useful if you have two config.yaml files, one for example being secret-config.yaml, and one being non-secret-config.yaml, and both wanted to add some extraConfig. If both these files had written a string value to extraConfig like we actually do using the | symbol in the YAML syntax, they would override instead of merge in a meaningful way. By introducing a key/value pair in between we can avoid this. That is the only purpose it serves though, so you can name it whatever you like. When the configuration snippets is executed listed under extraConfig, it will be done so in an alphabetical order based on the key name.

extraConfig:
  config1: |
    print("will execute first")
    print("PS: this is Python")
  config2: |
    print("will execute second")
3 Likes

I tried using an init container as you said. I can tell that /etc/jupyterhub custom is created, nothing is inside it. Maybe it cloned the files incorrectly?

hmm, hard to say what went wrong. Could you share the config you used?

@consideRatio thanks a lot for sharing all this knowledge! I will also try out this approach today :slight_smile:

1 Like

The more relevant part of config.yaml:

hub:
  # Clone custom JupyterHub templates into a volume
  initContainers:
    - name: git-clone-templates
      image: alpine/git
      args:
        - clone
          - --single-branch
          - --branch=master
          - --depth=1
          - --
        - https://github.com/LibreTexts/jupyterhub-templates.git
        - /etc/jupyterhub/custom
      securityContext:
        runAsUser: 0
      volumeMounts:
        - name: custom-templates
          mountPath: /etc/jupyterhub/custom
  extraVolumes:
    - name: custom-templates
      emptyDir: {}
  extraVolumeMounts:
    - name: custom-templates
      mountPath: /etc/jupyterhub/custom

  extraConfig:
    templates: |
      c.JupyterHub.template_paths = ['/etc/jupyterhub/custom/jupyterhub/templates']
    jupyterlab: |
      c.Spawner.cmd = ['jupyter-labhub']

Here’s the rest of it, if it’s applicable:

proxy:
  secretToken: <token>
  https:
    hosts:
      - domain.org
    letsencrypt:
      contactEmail: 'email.com'

singleuser:
  defaultUrl: "/lab"
  storage:
    capacity: 0.5G
  cpu:
    limit: 4
    guarantee: 0.5
  memory:
    limit: 8G
    guarantee: 1G
  image:
    name: libretexts/default-test
    tag: sagemath7 
  profileList:
    - display_name: "Default Environment"
      description: "With Python, R, Julia, Octave, and SageMath. Includes packages requested by the community."
      default: true
    - display_name: "Spark Environment"
      description: "The Jupyter Stacks spark image!"
      kubespawner_override:
        image: jupyter/all-spark-notebook:latest

auth:
  type: google
  admin:
    access: true
    users:
      - user@gmail.com
  whitelist:
    users:
      - user@gmail.com
  google:
    clientId: "client-id"
    clientSecret: "verysecret"
    callbackUrl: "https://domain.org/hub/oauth_callback"
    loginService: "Gmail"

I also tried @consideRatio’s solution with slightly different folder paths and reached a similar problem (although I may have botched the helm install of the chart, will try again soon). So it’s most likely a problem on my end?

There are some extra spaces in hub.initContainers.args (after clone command). Actually this should make git-clone-templates container fail and hub shouldn’t start at all. And also with that config you should set c.JupyterHub.template_paths to ['/etc/jupyterhub/custom']. Here is updated version of your config:

hub:
  # Clone custom JupyterHub templates into a volume
  initContainers:
    - name: git-clone-templates
      image: alpine/git
      args:
        - clone
        - --single-branch
        - --branch=master
        - --depth=1
        - --
        - https://github.com/LibreTexts/jupyterhub-templates.git
        - /etc/jupyterhub/custom
      securityContext:
        runAsUser: 0
      volumeMounts:
        - name: custom-templates
          mountPath: /etc/jupyterhub/custom
  extraVolumes:
    - name: custom-templates
      emptyDir: {}
  extraVolumeMounts:
    - name: custom-templates
      mountPath: /etc/jupyterhub/custom

  extraConfig:
    templates: |
      c.JupyterHub.template_paths = ['/etc/jupyterhub/custom']

Can you try this?

My bad, I initially wrote c.JupyterHub.template_paths = ['/etc/jupyterhub/custom']; I changed it to c.JupyterHub.template_paths = ['/etc/jupyterhub/custom/jupyterhub/templates'] while trying to debug. The extra spaces might have come from me copying from a backup file that wasn’t .yaml.

I tried using your config, nothing seems to be in /etc/jupyterhub/custom in the hub- pod still.

The custom folder seems to be mounted (although I’m not sure if that’s the right location):

$ kubectl exec hub-<name> -n jhub -ti bash
$ $ cat /etc/mtab | grep "custom"
/dev/mapper/VGsys-main /etc/jupyterhub/custom ext4 rw,relatime,errors=remount-ro,data=ordered 0 0

The hub pod is definitely running and I can access JupyterHub. So I do think that the initContainers did run or they somehow weren’t recognized.

I also tried disabling the firewall via sudo ufw disable, files still didn’t appear.

Yes it looks like initContainers were not started at all. Which version of JupyterHub chart are you using? Version 0.8.2 as from documentation? initContainers config is added into hub at version 0.9-38e1b89 (https://github.com/jupyterhub/zero-to-jupyterhub-k8s/pull/1274/). Sorry that I didn’t think about this before. Probably this is the issue. Can you try with latest development release (0.9-b63f5c9) of JupyterHub? Here you can find the all releases: https://jupyterhub.github.io/helm-chart/#development-releases-jupyterhub

But you have to also go through what is changed from 0.8.2 to 0.9-b63f5c9 (e.g. JupyterHub version is upgraded from 0.9.4 to 1.0.0) and update your configuration.

2 Likes

I should have paid attention to the JupyterHub version earlier. Thank you so much for helping, got it working now!