We’re starting this topic to capture / collect HPC-related Jupyter projects and resources. This is a broad topic and can include projects that work with JupyterHub (e.g. spawners that are commonly used in HPC contexts, services people find userful) or Jupyter notebook, JupyterLab, etc. (e.g. extensions that are commonly deployed for HPC users).
Feel free to contribute links to repos or projects that have to do with the intersection of Jupyter and HPC. Of course, if you have a new announcement of such a project, you can make a separate Discourse post about it! We encourage you to tag it with the “HPC” tag!
To start off, here are a few!:
- batchspawner: A JupyterHub spawner for that is designed for installation on clusters using batch scheduling software (e.g. Slurm).
- wrapspawner: Allows runtime configuration of JupyterHub spawners. The inspiration was to allow users to select from a range of pre-defined batch job profiles, but their operation is completely generic.
- jupyter-slurm-provisioner: Enables a user to launch a Jupyter kernel as a Slurm job, and connect to the kernel from a notebook server running on an interactive / login node.
- jupyter-lmod: Jupyter interactive notebook server extension that allows users to interact with environment modules before launching kernels. The extension use Lmod’s Python interface to accomplish module related task like loading, unloading, saving collection, etc.
- jupyterhub-announcement: JupyterHub service for easily managing announcements that can appear on JupyterHub pages, helpful for communicating with users about downtime, maintenance, issues, etc.