I am trying to run Jupyterhub in a cluster mode using a HA Load balancer configuration.
Since we will have the jupyterhub run individual spawners on each of the nodes or even on multiple nodes for the same user,
I would like to use some distributed storage like a hadoop or s3 where the spawners can spawn singe user server.
Reading about it I did find some material but that works only with Kubernetes spawner.
Is it possible to have a stand alone spawner which can use distributed storage for spawning single-user servers??
Right now I am able to do the S3 bucket & object management with my custom spawner but not able to spawn the single user server off the S3 storage.
Arguments: ('jupyterhub-singleuser --port=12345 --notebook-dir=s3a://jupyter/server/1234 --SingleUserNotebookApp.default_url=/lab
Spawn failed: Server at http://127.0.0.1:12345/jupyter/user/1234/ didn’t respond in 120 seconds
Let me know any ideas, suggestions or pointers on this one.