am getting “500 Internal server error” when starting Jupyterhub…
here’s some logs:
Oct 10 14:39:24 vlxhdpc10 jupyterhub: [I 2019-10-10 14:39:24.909 JupyterHub app:1673] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.4
Oct 10 14:39:24 vlxhdpc10 jupyterhub: [I 2019-10-10 14:39:24.909 JupyterHub app:1673] Using Spawner: sudospawner.spawner.SudoSpawner
Oct 10 14:39:24 vlxhdpc10 jupyterhub: [I 2019-10-10 14:39:24.913 JupyterHub app:1016] Loading cookie_secret from /etc/jupyterhub/jupyterhub_cookie_secret
Oct 10 14:39:24 vlxhdpc10 jupyterhub: [I 2019-10-10 14:39:24.930 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN
Oct 10 14:39:24 vlxhdpc10 jupyterhub: [W 2019-10-10 14:39:24.931 JupyterHub app:1173] No admin users, admin interface will be unavailable.
Oct 10 14:39:24 vlxhdpc10 jupyterhub: [W 2019-10-10 14:39:24.931 JupyterHub app:1174] Add any administrative users to `c.Authenticator.admin_users` in config.
Oct 10 14:39:24 vlxhdpc10 jupyterhub: [I 2019-10-10 14:39:24.931 JupyterHub app:1201] Not using whitelist. Any authenticated user will be allowed.
Oct 10 14:39:24 vlxhdpc10 jupyterhub: [I 2019-10-10 14:39:24.983 JupyterHub app:1855] Hub API listening on http://127.0.0.1:12000/hub/
Oct 10 14:39:24 vlxhdpc10 jupyterhub: [W 2019-10-10 14:39:24.984 JupyterHub proxy:565] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else...
Oct 10 14:39:24 vlxhdpc10 jupyterhub: [I 2019-10-10 14:39:24.984 JupyterHub proxy:567] Starting proxy @ http://:12001/
Oct 10 14:39:25 vlxhdpc10 jupyterhub: 14:39:25.554 - info: [ConfigProxy] Proxying http://*:12001 to (no default)
Oct 10 14:39:25 vlxhdpc10 jupyterhub: 14:39:25.556 - info: [ConfigProxy] Proxy API at http://0.0.0.0:12002/api/routes
Oct 10 14:39:25 vlxhdpc10 jupyterhub: [I 2019-10-10 14:39:25.646 JupyterHub proxy:370] Adding default route for Hub: / => http://127.0.0.1:12000
Oct 10 14:39:25 vlxhdpc10 jupyterhub: [I 2019-10-10 14:39:25.649 JupyterHub app:1912] JupyterHub is now running at http://:12001/
Oct 10 14:40:15 vlxhdpc10 jupyterhub: [W 2019-10-10 14:40:15.841 JupyterHub base:242] Invalid or expired cookie token
Oct 10 14:40:15 vlxhdpc10 jupyterhub: [I 2019-10-10 14:40:15.864 JupyterHub log:158] 200 GET /hub/login (@127.0.0.1) 24.62ms
Oct 10 14:40:22 vlxhdpc10 jupyterhub: [I 2019-10-10 14:40:22.764 JupyterHub base:499] User logged in: kuand
Oct 10 14:40:22 vlxhdpc10 jupyterhub: [I 2019-10-10 14:40:22.765 JupyterHub log:158] 302 POST /hub/login?next= -> /user/kuand/ (kuand@127.0.0.1) 50.36ms
Oct 10 14:40:22 vlxhdpc10 jupyterhub: [I 2019-10-10 14:40:22.793 JupyterHub log:158] 302 GET /user/kuand/ -> /hub/user/kuand/ (@127.0.0.1) 0.70ms
Oct 10 14:40:23 vlxhdpc10 jupyterhub: [W 2019-10-10 14:40:23.381 JupyterHub auth:642] Failed to open PAM session for kuand: [PAM Error 14] Cannot make/remove an entry for the specified session
Oct 10 14:40:23 vlxhdpc10 jupyterhub: [W 2019-10-10 14:40:23.381 JupyterHub auth:643] Disabling PAM sessions from now on.
Oct 10 14:40:24 vlxhdpc10 jupyterhub: [I 191010 14:40:23 mediator:89] Spawning /test/apps/anaconda3/bin/jupyterhub-singleuser --port=41438
Oct 10 14:40:24 vlxhdpc10 jupyterhub: Traceback (most recent call last):
Oct 10 14:40:24 vlxhdpc10 jupyterhub: File "/test/apps/anaconda3/bin/jupyterhub-singleuser", line 3, in <module>
Oct 10 14:40:24 vlxhdpc10 jupyterhub: from jupyterhub.singleuser import main
Oct 10 14:40:24 vlxhdpc10 jupyterhub: File "/test/apps/anaconda3/lib/python3.7/site-packages/jupyterhub/singleuser.py", line 34, in <module>
Oct 10 14:40:24 vlxhdpc10 jupyterhub: from notebook.notebookapp import (
Oct 10 14:40:24 vlxhdpc10 jupyterhub: File "/test/apps/anaconda3/lib/python3.7/site-packages/notebook/notebookapp.py", line 80, in <module>
Oct 10 14:40:24 vlxhdpc10 jupyterhub: from .log import log_request
Oct 10 14:40:24 vlxhdpc10 jupyterhub: File "/test/apps/anaconda3/lib/python3.7/site-packages/notebook/log.py", line 10, in <module>
Oct 10 14:40:24 vlxhdpc10 jupyterhub: from .prometheus.log_functions import prometheus_log_method
Oct 10 14:40:24 vlxhdpc10 jupyterhub: ModuleNotFoundError: No module named 'notebook.prometheus.log_functions'
Oct 10 14:40:24 vlxhdpc10 jupyterhub: [E 191010 14:40:24 mediator:43] mediator result: {'ok': False, 'error': 'Exited with status: 1'}
Oct 10 14:40:24 vlxhdpc10 jupyterhub: [E 2019-10-10 14:40:24.212 JupyterHub user:477] Unhandled error starting kuand's server: 'pid'
Oct 10 14:40:24 vlxhdpc10 jupyterhub: [E 2019-10-10 14:40:24.233 JupyterHub web:1788] Uncaught exception GET /hub/user/kuand/ (127.0.0.1)
Oct 10 14:40:24 vlxhdpc10 jupyterhub: HTTPServerRequest(protocol='http', host='vlxhdpc10:12001', method='GET', uri='/hub/user/kuand/', version='HTTP/1.1', remote_ip='127.0.0.1')
Oct 10 14:40:24 vlxhdpc10 jupyterhub: Traceback (most recent call last):
Oct 10 14:40:24 vlxhdpc10 jupyterhub: File "/test/apps/anaconda3/lib/python3.7/site-packages/tornado/web.py", line 1699, in _execute
Oct 10 14:40:24 vlxhdpc10 jupyterhub: result = await result
Oct 10 14:40:24 vlxhdpc10 jupyterhub: File "/test/apps/anaconda3/lib/python3.7/site-packages/jupyterhub/handlers/base.py", line 1052, in get
Oct 10 14:40:24 vlxhdpc10 jupyterhub: await self.spawn_single_user(user)
Oct 10 14:40:24 vlxhdpc10 jupyterhub: File "/test/apps/anaconda3/lib/python3.7/site-packages/jupyterhub/handlers/base.py", line 705, in spawn_single_user
Oct 10 14:40:24 vlxhdpc10 jupyterhub: timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future
Oct 10 14:40:24 vlxhdpc10 jupyterhub: File "/test/apps/anaconda3/lib/python3.7/site-packages/jupyterhub/handlers/base.py", line 626, in finish_user_spawn
Oct 10 14:40:24 vlxhdpc10 jupyterhub: await spawn_future
Oct 10 14:40:24 vlxhdpc10 jupyterhub: File "/test/apps/anaconda3/lib/python3.7/site-packages/jupyterhub/user.py", line 489, in spawn
Oct 10 14:40:24 vlxhdpc10 jupyterhub: raise e
Oct 10 14:40:24 vlxhdpc10 jupyterhub: File "/test/apps/anaconda3/lib/python3.7/site-packages/jupyterhub/user.py", line 409, in spawn
Oct 10 14:40:24 vlxhdpc10 jupyterhub: url = await gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
Oct 10 14:40:24 vlxhdpc10 jupyterhub: File "/test/apps/anaconda3/lib/python3.7/site-packages/sudospawner/spawner.py", line 93, in start
Oct 10 14:40:24 vlxhdpc10 jupyterhub: self.pid = reply['pid']
Oct 10 14:40:24 vlxhdpc10 jupyterhub: KeyError: 'pid'
Oct 10 14:40:24 vlxhdpc10 jupyterhub: [E 2019-10-10 14:40:24.245 JupyterHub log:150] {
Oct 10 14:40:24 vlxhdpc10 jupyterhub: "Upgrade-Insecure-Requests": "1",
Oct 10 14:40:24 vlxhdpc10 jupyterhub: "Cookie": "jupyterhub-hub-login=\"2|1:0|10:1570736422|20:jupyterhub-hub-login|44:NTNmNGRjYjBhMTg1NGFmYTg1MzIzYjUwMjc5ODFhZDk=|42781af779f5b8d0779147596f3fca1752f71c506ea1449f74fb62827cbcb6cf\"; _xsrf=2|0c3a49c5|110b91f3e1833fe0886782cf036276fd|1569953983; jupyterhub-session-id=cf36eda0979c4bddbb19bcd9c0fbee16",
Oct 10 14:40:24 vlxhdpc10 jupyterhub: "Connection": "close",
Oct 10 14:40:24 vlxhdpc10 jupyterhub: "Referer": "http://vlxhdpc10:12001/hub/login",
Oct 10 14:40:24 vlxhdpc10 jupyterhub: "Accept-Encoding": "gzip, deflate",
Oct 10 14:40:24 vlxhdpc10 jupyterhub: "Accept-Language": "en-US,en;q=0.5",
Oct 10 14:40:24 vlxhdpc10 jupyterhub: "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
Oct 10 14:40:24 vlxhdpc10 jupyterhub: "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0",
Oct 10 14:40:24 vlxhdpc10 jupyterhub: "Host": "vlxhdpc10:12001"
Oct 10 14:40:24 vlxhdpc10 jupyterhub: }
Oct 10 14:40:24 vlxhdpc10 jupyterhub: [E 2019-10-10 14:40:24.245 JupyterHub log:158] 500 GET /hub/user/kuand/ (kuand@127.0.0.1) 1428.31ms
anyone have any ideas?