Error starting Python3 kernel and permission error when listing kernelspec

JupyterHub
1

After a powercut, I have jupyterhub running again, but I cannot start the Python3 kernel. When opening an exisiting notebook, I receive a popup message:

Error Starting Kernel
Unhandled error
Unhandled Error

JupyterHub was setup in a virtual environment (/opt/jupyterhub/ - following the same setup as Part 1 here), and has been working without any issue for several months. I have tried to reinstall the kernel within this virtual environment, but I see the same error messages.

If I try to open a new Python3 notebook, I received the following popup message:

Error.
Unexpected error when saving file: folder/Untitled.ipynb [Errno 13] Permission denied ‘home/user/Office/Jupyter/Untitled.ipynb’

Followed by another popup message:

Launcher Error
Cannot read ‘path’ of undefined

If I run jupyterhub kernelspec list in the terminal, I get:

[I 2021-05-27 11:10:56.757 JupyterHub app:2349] Running JupyterHub version 1.3.0 [I 2021-05-27 11:10:56.757 JupyterHub app:2379] Using Authenticator: jupyterhub.auth.PAMAuthenticator-1.3.0 [I 2021-05-27 11:10:56.757 JupyterHub app:2379] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-1.3.0 [I 2021-05-27 11:10:56.758 JupyterHub app:2379] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-1.3.0 [I 2021-05-27 11:10:56.759 JupyterHub app:1420] Loading cookie_secret from /tmp/jupyterhub_cookie_secret [E 2021-05-27 11:10:56.769 JupyterHub app:2859] Traceback (most recent call last): File "/home/infuser/.local/lib/python3.8/site-packages/jupyterhub/app.py", line 2856, in launch_instance_async await self.initialize(argv) File "/home/infuser/.local/lib/python3.8/site-packages/jupyterhub/app.py", line 2391, in initialize self.init_db() File "/home/infuser/.local/lib/python3.8/site-packages/jupyterhub/app.py", line 1592, in init_db self.session_factory = orm.new_session_factory( File "/home/infuser/.local/lib/python3.8/site-packages/jupyterhub/orm.py", line 880, in new_session_factory check_db_revision(engine) File "/home/infuser/.local/lib/python3.8/site-packages/jupyterhub/orm.py", line 776, in check_db_revision with _temp_alembic_ini(engine.url) as ini: File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/infuser/.local/lib/python3.8/site-packages/jupyterhub/dbutil.py", line 74, in _temp_alembic_ini write_alembic_ini(alembic_ini, db_url) File "/home/infuser/.local/lib/python3.8/site-packages/jupyterhub/dbutil.py", line 38, in write_alembic_ini with open(alembic_ini, 'w') as f: PermissionError: [Errno 13] Permission denied: '/tmp/tmp52stgmqv/alembic.ini'

Clearly I have some kind of path issue (?), but I’m not sure what to do diagnose and hopefully fix this. What should I do next?

2

It’s very strange that jupyterhub wouldn’t be able to write a temporary file. Can you check the permissions of the jupyterhub user and if /tmp/tmp52stgmqv is in an environment variable anywhere and what the permissions on that directory are?

3

Hi minrk, thanks for your quick reply.

/tmp/tmp52stgmqv no longer exists, but here are the permissions of the directory:

infuser@inf-off-vm3:/tmp$ ls -la
total 176
drwxrwxrwt 43 root    root  4096 mag 27 13:18 .
drwxr-xr-x 20 root    root  4096 mag 27 13:16 ..
-rw-------  1 infuser users    0 mag 26 13:16 config-err-Eqzg5s
drwxrwxrwt  2 root    root  4096 mag 26 13:15 .font-unix
drwxrwxrwt  2 root    root  4096 mag 26 13:16 .ICE-unix
-rw-------  1 infuser sudo    65 mag 27 11:10 jupyterhub_cookie_secret
-rw-------  1 infuser users    0 mag 27 10:59 jupyterhub.sqlite
d-w-------  2 infuser users 4096 mag 27 10:14 pip-ephem-wheel-cache-7eyy261f
d-w-------  2 infuser users 4096 mag 27 10:20 pip-ephem-wheel-cache-hb71hi3v
d-w-------  2 infuser users 4096 mag 27 10:09 pip-ephem-wheel-cache-kfiplskz
d-w-------  2 infuser users 4096 mag 26 15:48 pip-ephem-wheel-cache-nf6o04ns
drwx------  2 root    root  4096 mag 26 15:41 pip-ephem-wheel-cache-wu_axz0s
d-w-------  2 infuser users 4096 mag 27 10:14 pip-install-947tq2is
d-w-------  2 infuser users 4096 mag 27 10:20 pip-install-94ijr9kg
d-w-------  2 infuser users 4096 mag 27 10:09 pip-install-or3zso0r
drwx------  2 root    root  4096 mag 26 15:41 pip-install-u_jlbre1
d-w-------  2 infuser users 4096 mag 26 15:48 pip-install-ya92uk72
d-w-------  2 infuser users 4096 mag 27 10:20 pip-req-tracker-1wls0xui
d-w-------  2 infuser users 4096 mag 27 10:14 pip-req-tracker-crlvfwvu
drwx------  2 root    root  4096 mag 26 15:41 pip-req-tracker-nj1pbw02
d-w-------  2 infuser users 4096 mag 26 15:48 pip-req-tracker-v3w0t35d
d-w-------  2 infuser users 4096 mag 27 10:09 pip-req-tracker-zwqtvv6j
d-w-------  2 infuser users 4096 mag 26 15:48 pip-unpack-d6bem83_
drwx------  2 infuser users 4096 mag 26 13:16 ssh-JbRnIt4bqCCI
drwx------  3 root    root  4096 mag 26 13:15 systemd-private-9d64b8ea507f486586da8c553f6333db-colord.service-5qmmKh
drwx------  3 root    root  4096 mag 27 13:18 systemd-private-9d64b8ea507f486586da8c553f6333db-fprintd.service-EdBZcj
drwx------  3 root    root  4096 mag 26 13:15 systemd-private-9d64b8ea507f486586da8c553f6333db-ModemManager.service-26Wtnj
drwx------  3 root    root  4096 mag 26 13:15 systemd-private-9d64b8ea507f486586da8c553f6333db-switcheroo-control.service-Cy8Dhh
drwx------  3 root    root  4096 mag 26 13:15 systemd-private-9d64b8ea507f486586da8c553f6333db-systemd-logind.service-cpNyzi
drwx------  3 root    root  4096 mag 26 13:15 systemd-private-9d64b8ea507f486586da8c553f6333db-systemd-resolved.service-4RW88g
drwx------  3 root    root  4096 mag 26 13:15 systemd-private-9d64b8ea507f486586da8c553f6333db-systemd-timesyncd.service-Qig81f
drwx------  3 root    root  4096 mag 26 13:15 systemd-private-9d64b8ea507f486586da8c553f6333db-traefik.service-7seu7i
drwx------  3 root    root  4096 mag 26 13:15 systemd-private-9d64b8ea507f486586da8c553f6333db-upower.service-hjotGg
drwx------  2 infuser users 4096 mag 26 16:37 Temp-40ae6a5c-3a20-453a-b952-3a9034d9f6a5
drwx------  2 infuser users 4096 mag 26 16:37 Temp-a66fe5ed-ef1a-4242-a0c0-17ca460652f4
drwxrwxrwt  2 root    root  4096 mag 26 13:15 .Test-unix
drwx------  3 root    root  4096 mag 27 10:29 tmp7z8lqwcd_kernels
d---------  2 infuser users 4096 mag 27 10:21 tmpd_4gqy3d_kernels
drwx------  2 root    root  4096 mag 27 10:24 tmpjz19oc54_kernels
drwx------  2 root    root  4096 mag 27 10:29 tmpk5p1ccq4_kernels
d---------  2 infuser users 4096 mag 27 10:29 tmpn96_dica_kernels
d---------  2 infuser users 4096 mag 27 10:30 tmpqlpsqjmz_kernels
d---------  2 infuser users 4096 mag 27 10:21 tmpt527t_c5_kernels
drwx------  2 infuser users 4096 mag 27 10:46 tracker-extract-files.1000
drwxrwxrwt  2 root    root  4096 mag 26 13:15 .X11-unix
drwxrwxrwt  2 root    root  4096 mag 26 13:15 .XIM-unix

And here is the user:

infuser@inf-off-vm3:~$ id infuser
uid=1000(infuser) gid=100(users) groups=100(users),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),120(lpadmin),131(lxd),132(sambashare)