Failed to start server or login as other users.(Jupyterhub run as daemon)

tomf · February 24, 2025, 3:02pm

I have installed jupyterhub and run it as a daemon in a VM (ubuntu) with a systemuser account. Login with the same systemuser account have no issue. But it fail when I try to log in with other user account (user1, user2 or user3).

I have set the jupyterhub_config.py as follow:

c.Authenticator.whitelist = {'systemuser', 'user1', 'user2', 'user3'}
c.Authenticator.allowed_users = {'systemuser', 'user1', 'user2', 'user3'}
c.PAMAuthenticator.allowed_users = {'systemuser', 'user1', 'user2', 'user3'}
c.LocalAuthenticator.create_system_users = True
c.JupyterHub.authenticator_class = 'jupyterhub.auth.PAMAuthenticator'
c.JupyterHub.spawner_class = 'jupyterhub.spawner.LocalProcessSpawner'

and config the jupyterhub_service as folllow:

/etc/systemd/system/jupyterhub.service

[Unit]
Description=JupyterHub
After=network.target

[Service]
User=systemuser
ExecStart=/usr/local/bin/jupyterhub --config /home/systemuser/jupyterhub_config.py
WorkingDirectory=/home/systemuser
Restart=always
Environment=PATH=/usr/local/bin:/usr/bin:/bin
Environment=PYTHONPATH=/usr/local/lib/python3.x/site-packages

[Install]
WantedBy=multi-user.target

Feb 24 10:04:56 labsandbox jupyterhub[965]: Feb 24 10:06:28 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: Feb 24 10:06:29 labsandbox jupyterhub[965]: [E 2025-02-24 10:04:56.459 JupyterHub log:192] 500 POST /hub/api/users/user2/servers/?_xsrf=[secret] (ctc@::ffff:10.12.133.52) 105.18ms
[I 2025-02-24 10:06:28.998 JupyterHub provider:661] Creating oauth client jupyterhub-user-user2
[I 2025-02-24 10:06:29.016 JupyterHub spawner:1837] Spawning jupyterhub-singleuser
[E 2025-02-24 10:06:29.025 JupyterHub user:1007] Unhandled error starting user2’s server: Exception occurred in preexec_fn.
Traceback (most recent call last):
File “/usr/local/lib/python3.8/dist-packages/jupyterhub/user.py”, line 920, in spawn
await asyncio.wait_for(f, timeout=spawner.start_timeout)
File “/usr/lib/python3.8/asyncio/tasks.py”, line 494, in wait_for
return fut.result()
File “/usr/local/lib/python3.8/dist-packages/jupyterhub/spawner.py”, line 1847, in start
self.proc = Popen(cmd, **popen_kwargs)
File “/usr/lib/python3.8/subprocess.py”, line 858, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File “/usr/lib/python3.8/subprocess.py”, line 1705, in _execute_child
raise child_exception_type(err_msg)
subprocess.SubprocessError: Exception occurred in preexec_fn.

[E 2025-02-24 10:06:29.051 JupyterHub web:1875] Uncaught exception POST /hub/api/users/ai_user2/servers/?_xsrf=MnwxOjB8MTA6MTc0MDM4OTA5Nnw1Ol94c3JmfDg4Oll6ZzVNREZtTVdRMFpqUmlORE5rTURreU1EUm1PVFZoTXpBeVlqa3dOVGc2TURoaU5qWmhORGM0WTJJeE5EQTNaamcyWVRWbU1XSXlObUV6T0RrNE1ERT18MGFlYTA1YjA2M2JlZWVkZDhhOTI2MmYzYmRkYTI3MjU5MTNjYWRkYWY0MzcyZTViMzQ4YWYzNWU4YzA5MDgwNw (::ffff:10.12.133.52)
HTTPServerRequest(protocol=‘http’, host=‘10.12.196.103:8000’, method=‘POST’, uri=‘/hub/api/users/ai_user2/servers/?_xsrf=MnwxOjB8MTA6MTc0MDM4OTA5Nnw1Ol94c3JmfDg4Oll6ZzVNREZtTVdRMFpqUmlORE5rTURreU1EUm1PVFZoTXpBeVlqa3dOVGc2TURoaU5qWmhORGM0WTJJeE5EQTNaamcyWVRWbU1XSXlObUV6T0RrNE1ERT18MGFlYTA1YjA2M2JlZWVkZDhhOTI2MmYzYmRkYTI3MjU5MTNjYWRkYWY0MzcyZTViMzQ4YWYzNWU4YzA5MDgwNw’, version=‘HTTP/1.1’, remote_ip=‘::ffff:10.12.133.52’)
Traceback (most recent call last):
File “/usr/local/lib/python3.8/dist-packages/tornado/web.py”, line 1790, in _execute
result = await result
File “/usr/local/lib/python3.8/dist-packages/jupyterhub/apihandlers/users.py”, line 666, in post
await self.spawn_single_user(user, server_name, options=options)
File “/usr/local/lib/python3.8/dist-packages/jupyterhub/handlers/base.py”, line 1215, in spawn_single_user
await gen.with_timeout(
File “/usr/local/lib/python3.8/dist-packages/jupyterhub/handlers/base.py”, line 1122, in finish_user_spawn
await spawn_future
File “/usr/local/lib/python3.8/dist-packages/jupyterhub/user.py”, line 1021, in spawn
raise e
File “/usr/local/lib/python3.8/dist-packages/jupyterhub/user.py”, line 920, in spawn
await asyncio.wait_for(f, timeout=spawner.start_timeout)
File “/usr/lib/python3.8/asyncio/tasks.py”, line 494, in wait_for
return fut.result()
File “/usr/local/lib/python3.8/dist-packages/jupyterhub/spawner.py”, line 1847, in start
self.proc = Popen(cmd, **popen_kwargs)
File “/usr/lib/python3.8/subprocess.py”, line 858, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File “/usr/lib/python3.8/subprocess.py”, line 1705, in _execute_child
raise child_exception_type(err_msg)
subprocess.SubprocessError: Exception occurred in preexec_fn.

[E 2025-02-24 10:06:29.052 JupyterHub log:184] {
“X-Forwarded-Host”: “10.12.196.103:8000”,
“X-Forwarded-Proto”: “http”,
“X-Forwarded-Port”: “8000”,
“X-Forwarded-For”: “::ffff:10.12.133.52”,
“Cookie”: “jupyterhub-hub-login=[secret]; _xsrf=[secret]; jupyterhub-session-id=[secret]”,
“Accept-Language”: “en-US,en;q=0.9”,
“Accept-Encoding”: “gzip, deflate”,
“Referer”: “http://10.12.196.103:8000/hub/admin”,
“Origin”: “http://10.12.196.103:8000”,
“Content-Type”: “application/json”,
“Accept”: “application/jupyterhub-pagination+json”,
“User-Agent”: “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36”,
“Content-Length”: “0”,
“Connection”: “keep-alive”,
“Host”: “10.12.196.103:8000”
}

mahendrapaipuri · February 24, 2025, 8:45pm

To spawn single user servers, the user running JupyterHub, systemuser in your case, needs privileges as LocalProcessSpawner spawns the server as the target user. Please add this user to sudoers list and it should work. I see in your config, you have c.LocalAuthenticator.create_system_users = True but you are using jupyterhub.auth.PAMAuthenticator. If you want to use LocalAuthenticator, you need to change it to c.JupyterHub.authenticator_class = 'jupyterhub.auth.LocalAuthenticator'. To be able to create system users dynamically, systemuser needs privileges too.

tomf · February 25, 2025, 1:27am

Thanks for your reply. I have made the change as per recommended: but it still doesn’t work.

c.JupyterHub.authenticator_class = 'jupyterhub.auth.LocalAuthenticator'

When I used ‘sudo jupyterhub’, all users can log in without problem.

However, I prefer to run it as daemon on boot, In this case, only the systemuser can login, while other users (user1, user2 or user3) can not login.


sudo systemctl daemon-reload
sudo systemctl enable jupyterhub
sudo systemctl start jupyterhub

Looking forward to any suggestion.

tomf · February 25, 2025, 3:12am

Found the solution.

change the user to root in /etc/systemd/system/jupyterhub.service will work

[Service]
User=systemuser

Topic		Replies	Views
JupyterHub doesn't allow me to login as another user JupyterHub	2	1271	February 25, 2025
[PAM Error 7] Authentication failure JupyterHub	1	2103	October 12, 2023
Login of os user to jupyterhub fails with SudoSpawner JupyterHub jupyterhub	1	805	April 1, 2022
I can't login my JupyterHub even once JupyterHub	3	2652	August 11, 2023
Unable to login to Jupyterhub Server JupyterHub	3	70	November 28, 2024

Failed to start server or login as other users.(Jupyterhub run as daemon)

Related topics