Spawn failed: Server at http://127.0.0.1:39145/user/sramineni/ didn't respond in 300 seconds

JupyterHub
, , ,
1

[I 2021-12-28 16:56:30.168 JupyterHub app:2766] Running JupyterHub version 2.1.0.dev
[I 2021-12-28 16:56:30.169 JupyterHub app:2797] Using Authenticator: jupyterhub.auth.PAMAuthenticator-2.1.0.dev
[I 2021-12-28 16:56:30.169 JupyterHub app:2797] Using Spawner: jupyterhub.spawner.SimpleLocalProcessSpawner-2.1.0.dev
[I 2021-12-28 16:56:30.169 JupyterHub app:2797] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-2.1.0.dev
[I 2021-12-28 16:56:30.177 JupyterHub app:1606] Loading cookie_secret from /home/ec2-user/jupyterhub/jupyterhub_cookie_secret
[I 2021-12-28 16:56:30.314 JupyterHub proxy:496] Generating new CONFIGPROXY_AUTH_TOKEN
[I 2021-12-28 16:56:30.327 JupyterHub app:1925] Not using allowed_users. Any authenticated user will be allowed.
[I 2021-12-28 16:56:30.357 JupyterHub app:2838] Initialized 0 spawners in 0.003 seconds
[W 2021-12-28 16:56:30.361 JupyterHub proxy:688] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else…
[I 2021-12-28 16:56:30.361 JupyterHub proxy:691] Starting proxy @ http://:8000
16:56:30.506 [ConfigProxy] info: Proxying http://*:8000 to (no default)
16:56:30.513 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes
16:56:30.719 [ConfigProxy] info: 200 GET /api/routes
[I 2021-12-28 16:56:30.719 JupyterHub app:3084] Hub API listening on http://127.0.0.1:8081/hub/
16:56:30.722 [ConfigProxy] info: 200 GET /api/routes
[I 2021-12-28 16:56:30.722 JupyterHub proxy:431] Adding route for Hub: / => http://127.0.0.1:8081
16:56:30.725 [ConfigProxy] info: Adding route / → http://127.0.0.1:8081
16:56:30.726 [ConfigProxy] info: Route added / → http://127.0.0.1:8081
16:56:30.726 [ConfigProxy] info: 201 POST /api/routes/
[I 2021-12-28 16:56:30.727 JupyterHub app:3150] JupyterHub is now running at http://:8000
[I 2021-12-28 16:56:39.713 JupyterHub log:189] 200 GET /hub/spawn-pending/sramineni (sramineni@::ffff:10.81.59.60) 73.71ms
[I 2021-12-28 16:56:41.943 JupyterHub roles:477] Adding role server to token: <APIToken(‘713a…’, user=‘sramineni’, client_id=‘jupyterhub’)>
[I 2021-12-28 16:56:41.965 JupyterHub provider:607] Creating oauth client jupyterhub-user-sramineni
[W 2021-12-28 16:56:42.012 JupyterHub auth:1107] Failed to open PAM session for sramineni: [PAM Error 14] Cannot make/remove an entry for the specified session
[W 2021-12-28 16:56:42.012 JupyterHub auth:1108] Disabling PAM sessions from now on.
[I 2021-12-28 16:56:42.031 JupyterHub spawner:1526] Spawning jupyterhub-singleuser
[I 2021-12-28 16:56:42.932 JupyterHub log:189] 302 GET /hub/spawn/sramineni → /hub/spawn-pending/sramineni (sramineni@::ffff:10.81.59.60) 1004.26ms
[I 2021-12-28 16:56:43.261 JupyterHub pages:400] sramineni is pending spawn
[I 2021-12-28 16:56:43.269 JupyterHub log:189] 200 GET /hub/spawn-pending/sramineni (sramineni@::ffff:10.81.59.60) 12.34ms
[I 2021-12-28 16:56:43.494 SingleUserLabApp mixins:615] Starting jupyterhub single-user server version 2.1.0.dev
[I 2021-12-28 16:56:43.494 SingleUserLabApp mixins:630] Extending jupyterlab.labhubapp.SingleUserLabApp from jupyterlab 3.2.5
[I 2021-12-28 16:56:43.495 SingleUserLabApp mixins:630] Extending jupyter_server.serverapp.ServerApp from jupyter_server 1.13.1
[I 2021-12-28 16:56:43.537 SingleUserLabApp manager:345] jupyterlab | extension was successfully linked.
[W 2021-12-28 16:56:43.552 SingleUserLabApp manager:349] No such notebook dir: ‘/notebooks’
[C 2021-12-28 16:56:43.561 SingleUserLabApp application:90] Bad config encountered during initialization: No such notebook dir: ‘/notebooks’
ERROR:asyncio:Task exception was never retrieved
future: <Task finished coro=<BaseHandler.spawn_single_user() done, defined at /home/ec2-user/jupyterhub/jupyterhub/handlers/base.py:831> exception=HTTPError()>
Traceback (most recent call last):
File “/home/ec2-user/jupyterhub/jupyterhub/handlers/base.py”, line 1033, in spawn_single_user
timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future
tornado.util.TimeoutError: Timeout

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home/ec2-user/jupyterhub/jupyterhub/handlers/base.py”, line 1069, in spawn_single_user
% (status, spawner._log_name),
tornado.web.HTTPError: HTTP 500: Internal Server Error (Spawner failed to start [status=1]. The logs for sramineni may contain details.)
[I 2021-12-28 16:59:53.518 JupyterHub pages:400] sramineni is pending spawn
[W 2021-12-28 17:01:24.028 JupyterHub user:812] sramineni’s server never showed up at http://127.0.0.1:39145/user/sramineni/ after 300 seconds. Giving up.

Common causes of this timeout, and debugging tips:

1. The server didn't finish starting,
   or it crashed due to a configuration issue.
   Check the single-user server's logs for hints at what needs fixing.
2. The server started, but is not accessible at the specified URL.
   This may be a configuration issue specific to your chosen Spawner.
   Check the single-user server logs and resource to make sure the URL
   is correct and accessible from the Hub.
3. (unlikely) Everything is working, but the server took too long to respond.
   To fix: increase `Spawner.http_timeout` configuration
   to a number of seconds that is enough for servers to become responsive.

[E 2021-12-28 17:01:24.072 JupyterHub gen:624] Exception in Future <Task finished coro=<BaseHandler.spawn_single_user..finish_user_spawn() done, defined at /home/ec2-user/jupyterhub/jupyterhub/handlers/base.py:935> exception=TimeoutError(“Server at http://127.0.0.1:39145/user/sramineni/ didn’t respond in 300 seconds”)> after timeout
Traceback (most recent call last):
File “/usr/local/lib64/python3.7/site-packages/tornado/gen.py”, line 618, in error_callback
future.result()
File “/home/ec2-user/jupyterhub/jupyterhub/handlers/base.py”, line 942, in finish_user_spawn
await spawn_future
File “/home/ec2-user/jupyterhub/jupyterhub/user.py”, line 792, in spawn
await self._wait_up(spawner)
File “/home/ec2-user/jupyterhub/jupyterhub/user.py”, line 836, in _wait_up
raise e
File “/home/ec2-user/jupyterhub/jupyterhub/user.py”, line 807, in _wait_up
http=True, timeout=spawner.http_timeout, ssl_context=ssl_context
File “/home/ec2-user/jupyterhub/jupyterhub/utils.py”, line 246, in wait_for_http_server
timeout=timeout,
File “/home/ec2-user/jupyterhub/jupyterhub/utils.py”, line 189, in exponential_backoff
raise asyncio.TimeoutError(fail_message)
concurrent.futures._base.TimeoutError: Server at http://127.0.0.1:39145/user/sramineni/ didn’t respond in 300 seconds

[I 2021-12-28 17:01:24.074 JupyterHub log:189] 200 GET /hub/api/users/sramineni/server/progress (sramineni@::ffff:10.81.59.60) 280414.46ms
[I 2021-12-28 17:01:24.075 JupyterHub log:189] 200 GET /hub/api/users/sramineni/server/progress (sramineni@::ffff:10.81.59.60) 89864.92ms
17:01:30.731 [ConfigProxy] info: 200 GET /api/routes
17:06:30.731 [ConfigProxy] info: 200 GET /api/routes

2

Hi! Please could you:

  • Tell us how you installed JupyterHub and related components, ideally provide enough information for someone to reproduce your problem including versions
  • Show us your full JupyterHub configuration with secrets redacted
3

i have installed jupyterhub using pip3 in amazon ec2 instance.
i am getting the below error while starting the server. can you please help me. do i need to install notebook as well? and also please share me the jupyterhub configuration file related to spawner chnages.

tornado.web.HTTPError: HTTP 500: Internal Server Error (Spawner failed to start [status=1]. The logs for sramineni may contain details.)
[W 2021-12-29 07:59:14.438 JupyterHub user:812] sramineni’s server never showed up at http://127.0.0.1:35633/user/sramineni/ after 300 seconds. Giving up.

Common causes of this timeout, and debugging tips:

1. The server didn't finish starting,
   or it crashed due to a configuration issue.
   Check the single-user server's logs for hints at what needs fixing.
2. The server started, but is not accessible at the specified URL.
   This may be a configuration issue specific to your chosen Spawner.
   Check the single-user server logs and resource to make sure the URL
   is correct and accessible from the Hub.
3. (unlikely) Everything is working, but the server took too long to respond.
   To fix: increase `Spawner.http_timeout` configuration
   to a number of seconds that is enough for servers to become responsive.
4

JupyterHub managers your singleuser servers (e.g. JupyterLab, notebook), but they’re not automatically installed since they can be run from different virtual environments, on on remote servers, depending on the spawner you’ve configured. That’s why it would be helpful to see your configuration.

If you’re not familiar with JupyterHub I recommend following The Littlest JupyterHub — The Littlest JupyterHub v0.1 documentation instead until you’re more familiar with how it works.