I’ve been toying around with setting up a JupyterHub deployment for a while but many of the technologies involved are not super familiar to me. I’m perfectly comfortable with my local jupyter lab setup but the complexity of JupyterHub seems on another level. I thought I might describe what I’d like to be able to do and see if people thought JupyterHub was the appropriate solution or if it was overkill.
Currently, I run Jupyer Lab locally from my workstation, but I sometimes run into situations where I’d like more power. I’ve considered migrating my set up to an EC2 instance, but would really like to be able to spin up instances on demand when the need arises and have them shut down once I close that particular project. I could stage a number of different EC2 instances of different sizes but that seems awkward. Also, continuously running a kubernetes server on EC2 or using EKS seems like overkill. It would be more justifiable if I got more people on my team using it, but I’m not at that point yet.
I think for my own use case the ideal would be to run JupyterLab from my workstation or a single EC2 instance in such a way that I could use remote kernels that launch different sized EC2 instances in a similar way to the way I can connect different notebooks to different kernels or conda environments. I’ve been intrigued by this description of how Harry’s approached the problem as well as the cloudJHub implementation at Harvard, but again those are both pushing me out of my comfort zone a bit.
I’m happy to try to learn more about some of these technologies, but have trouble prioritizing which ones. Any advice or suggestions? Thanks!