We are using containerized jupyterhub with individual containers spawned for individual users set up in the conventional way (I think). A couple of times, say 3 times over a year, a users container gets corrupted or affected in some way such that they cannot login. When they attempt they get a “Clear Workspace or Keep Waiting” dialog, which doesn’t seem to do anything. Eventually jupyter-lab comes up but it is completely empty, no files, no launchers, just the empty frame.
I can see the users spawned container running, and I am able to docker exec into it. All the files appear to be there and intact. I don’t see anything suspicious in the jupyterhub logs.
Is there a log file or debug procedure that I can use to track down what is happening? I’m running out of ideas at this point.