Cleanest way to isolate a z2jh user (or group? :-) on their own hardware?

I have a destructively productive scientist, who’s many and heavy training and model runs have a tendency to kill k8s nodes and kernels in a variety of ways, both swift and lingering.

“THE LAZY DEVOPS 3 LAWS OF KEV”
DEFINITELY don’t put your reverse proxy on the same metal as kev
PROBABLY don’t put your web services on the same metal as kev
MAYBE you don’t even want other users on the same metal as kev? :face_with_raised_eyebrow:

Congrats kev, you stand alone, like literally, we need your hw to stand alone :rofl:

Ideally, I’d like kev (and now that we’ve identified the pattern, possibly some other scientists too with heavier ml workloads) to have access to the same 4 profile flavors we make available to everyone else. Ideally ideally, I’d love this to be hella kiss…

I was thinking maybe a starting point I’m going to explore is modifying the kubespawner args from a py auth hook to add an anti-affinity (for all other pods and services? not sure if that’s a writable anti-affinity, tbd) if a user is in a set, starting with… the_cheese_stands_alone = { "kev" }

Tips and starting ideas welcome, or ya know, maybe just whole formed solutions yuvi dropped on you lol :see_no_evil::pray:t4:

-Seth