In Pangeo, we run jupyterhub and binder clusters using kubernetes on several different clouds. We often use dask_kuberentes to launch additional dask worker pods from our notebooks.
Both administrators and users are generally very curious about the status of the cluster as a whole. They would like to know
- What is the status of my dask pods?
- How many other users are on the cluster?
- What is the status of the VM nodes and the pod distribution among them?
- How much is it costing?
This information is available to admins via kubectl or the cloud console. But what if we could monitor it directly from our jupyterlab window (or, alternatively, from the jupyterhub interface)? This would be valuable for debugging but also for education. Lots of people are just curious about how cloud works. HPC users are used to being able to query the cluster load and job queue, and expect similar information to be available in the cloud.
Perhaps some tools already exists for this purpose that could be plugged in to meet this need.