Per-class kernels?

One thought we were having is whether it make sense to create special kernels tied to specific conda environments, for courses. Our thought is to have one hub but for a class that might need one set of packages (geopandas or more-accurately fiona is currently kind of annoyingly finicky about precise dependencies), we would have a custom kernel with that conda env. Does anybody have experiences with that? Additionally, could specific kernels be used to select hardware requirements as well … I am thinking here about GPUs. If we have a couple of notes with GPU and tensorflow, could they be used only when selecting a GPU based python kernel? Or for example a class might need the a kernel to run on a subset of machines which mounts a disk with a large data set.
Does this make sense?

For a US university that is a client of my company we operate a multi hub setup on a shared kubernetes cluster. This way each course gets its own JupyterHub and can set installed packages, resources and enrolled students independently. While at the same time sharing a pool of machines to increase utilisation. This setup works very well.

I just spotted this today, an example of bundling an environment in an electron app (cf something like nteract) coupled to an online course server with an anaconda environment:



I haven’t had chance to fully grok it yet/appreciate what the components, how they interact, how the approach affects course management, student workflow and experience, etc, let alone try it out myself.

But it maybe relevant to this thread, and if anyone is familiar with it they could perhaps give a quick overview?

–tony

1 Like

I could see that might be a good fit for some cases. One thing that worries me is administering multiple hub interfaces. Also enrolling students independently (in my view) is a bug rather than a feature. One of the great pluses of our otherwise infuriating LMS (blackboard) is that as a faculty member, I don’t have to deal with enrollment … the registrar and help desk can manage that. My question still stands though, can one use the kernel choice to (1) specify a software environment and (2) specify a selection of machines? I am pretty sure 1 should be possible.

Definitely interesting. I know Netflix had some nteract based electron standalone app they were demoing at JupyterCon but I can’t seem to find the project location.

The nteract project is at https://nteract.io/ which is the project Netflix supports and contributes to. We experimented with building conda environments to bundle with nteract. Stalled because it never got integrated with the desktop app (Tim didn’t finish what he started :-/)

You can install multiple kernels in multiple conda/pip environments and select them through the dropdown menu when you launch the kernel. You can’t choose the resources through the kernel menu as your container is already running by the time you get to the kernel choice menu.

However I’d use the profile_list option of KubeSpawner to select what resources and what image to use when a person logs in.