Kubespawner_override tolerations syntax is not great IMO

JupyterHub Zero to JupyterHub on Kubernetes
1

Disclaimer. I’m new at this! I was looking to add an extra toleration to a couple of profiles. Much to my dismay if I want to have a custom set of tolerations for a profile I need to list out all of them that are needed and I’m going to end up repeating myself alot. I’m surprised there isn’t an equivalent: extraTolerations setting unless I’m missing something. (Please:-))

 kubespawner_override:
        image: jupyter/scipy-notebook:latest
        tolerations:
        - key: blah.tld/mem
          operator: Equal
          value: high
          effect: NoSchedule
        - key: hub.jupyter.org/dedicated
          operator: Equal
          value: user
          effect: NoSchedule
        - key: kubernetes.azure.com/scalesetpriority
          operator: Equal
          value: spot
          effect: NoSchedule
2

There’s singleuser.extraTolerations - see [Configuration Reference — Zero to JupyterHub with Kubernetes documentation]. There’s a similar one for the prePuller which you might also have to set.

I hope this is useful, and thank you for opening an informative topic here :slight_smile:

3

Thanks! I’ve seen those, but what I’d like to do is for each profile add an extra toleration (for example: “blah.tld/mem” Equal high) as opposed to overriding all of the tolerations specified in singleUser.extraTolerations. What I’ve found is that when you use the kubespawner override it overrides all of the tolerations specified in singleUser.extraTolerations meaning any common tolerations across all profiles need to be repeated over and over again.

4

Aaah, that’s a very good use case. Unfortunately I don’t think there’s a way to do that yet :frowning:

5

Regarding kubespawner_override, I agree, it is very problematic. I have already opined that we should not make it merge options etc because it would be to confusing as sometimes you want it to override and other times you want it to merge dictionaries or lists. Due to this, my opinion is that the core problem is that kubespawner_override doesn’t meet all the needs, and it would be good to have something to complement it.

Btw, profile_list and its override setting etc is perhaps relevant to upstream to the Spawner base class rather than retain specifically for the KubeSpawner among all various spawners out there. It is a separate discussion though.

1 Like
6

@jabbera and @yuvipanda, while it is not possible with kubespawner_override to do a merge or insert etc, it is possible with logic I describe in this how-to post:

7

@consideRatio, thanks! This example is great!

1 Like
8

Closing the loop on this. I ended up hosting nodes with different values of the “user” value for the stock taint: hub.jupyter.org/node-purpose=user. (Example: gpu, peered, whatever property described what makes those nodes special). I wrote a custom form that decoupled the image from the node properties. Basically my question ended up being moot:-)

1 Like