# Jupyterhub single user sever not starting immediately on Kubernetes cluster **URL:** \ **Category:** JupyterHub\ **Created:** [September 7, 2023, 3:37pm UTC](https://discourse.jupyter.org/t/jupyterhub-single-user-sever-not-starting-immediately-on-kubernetes-cluster/21304 "2023-09-07T15:37:02Z")\ **Posts on this page:** 5\ **Page:** 1

**Author:** ![srujan\_gajula](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/srujan_gajula/32/10958_2.png) [@srujan\_gajula](https://discourse.jupyter.org/u/srujan_gajula)\ **Post date:** [September 7, 2023, 3:37pm UTC](https://discourse.jupyter.org/t/jupyterhub-single-user-sever-not-starting-immediately-on-kubernetes-cluster/21304/1 "2023-09-07T15:37:02Z")

We’ve deployed Jupyterhub on Kubernetes cluster. Followed from [Optimizations — Zero to JupyterHub with Kubernetes documentation](https://z2jh.jupyter.org/en/latest/administrator/optimization.html) for better optimization(used continues image puller, user place holders and user schedulers). When a user is trying to create a server to launch a jupyter lab instance , the server basically Kubernetes pod not starting immediately it is taking 35sec to start the server even we’ve user place holders placed. Any suggestions to start the server immediate without user wait time. Thank you ---

**Author:** ![manics](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/manics/32/85_2.png) [@manics](https://discourse.jupyter.org/u/manics)\ **Post date:** [September 7, 2023, 5:18pm UTC](https://discourse.jupyter.org/t/jupyterhub-single-user-sever-not-starting-immediately-on-kubernetes-cluster/21304/2 "2023-09-07T17:18:16Z")

Have you tried looking at the k8s events and logs to see where the delay is occurring? Do you see the same behaviour with other pods, or only those started by JupyterHub? ---

**Author:** ![srujan\_gajula](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/srujan_gajula/32/10958_2.png) [@srujan\_gajula](https://discourse.jupyter.org/u/srujan_gajula)\ **Post date:** [September 8, 2023, 2:40pm UTC](https://discourse.jupyter.org/t/jupyterhub-single-user-sever-not-starting-immediately-on-kubernetes-cluster/21304/3 "2023-09-08T14:40:10Z")

Thank you for response. Create server “hub/api/users/server\_name/server” always giving 202 status code instead 201 (which start server immediately) Here are events did not see much difference Events: │ │ Type Reason Age From Message │ │ ---- ------ ---- ---- ------- │ │ Warning FailedScheduling 42s jupyterhub-user-scheduler 0/8 nodes are available: not able to place differant reason │ │ Normal Scheduled 40s jupyterhub-user-scheduler Successfully assigned jupyter-xxx/jupyter-xxxx to ip-xxx.compute.internal │ │ Normal Pulled 39s kubelet Container image “jupyterhub/k8s-network-tools:2.0.0” already presen │ │ t on machine │ │ Normal Created 39s kubelet Created container block-cloud-metadata │ │ Normal Started 39s kubelet Started container block-cloud-metadata │ │ Normal Pulled 38s kubelet Container image “xxxxxxx” already present on machine │ │ Normal Created 38s kubelet Created container notebook │ │ Normal Started 38s kubelet Started container notebook Here are logs Defaulted container “notebook” out of: notebook, block-cloud-metadata (init) [I 2023-09-08 14:15:02.972 SingleUserLabApp mixins:609] Starting jupyterhub single-user server version 3.1.1 [I 2023-09-08 14:15:02.972 SingleUserLabApp mixins:623] Extending jupyterlab.labhubapp.SingleUserLabApp from jupyterlab 3.6.3 [I 2023-09-08 14:15:02.972 SingleUserLabApp mixins:623] Extending jupyter\_server.serverapp.ServerApp from jupyter\_server 2.5.0 [W 2023-09-08 14:15:03.006 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [I 2023-09-08 14:15:03.417 SingleUserLabApp utils:78] Package xxx\_extension took 0.3978s to import [W 2023-09-08 14:15:03.417 SingleUserLabApp utils:91] A `_jupyter_server_extension_points` function was not found in XXX\_extension. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server. [I 2023-09-08 14:15:04.239 SingleUserLabApp utils:78] Package jupyter\_env\_whitelist took 0.8215s to import [I 2023-09-08 14:15:04.244 SingleUserLabApp utils:78] Package jupyter\_server\_fileid took 0.0042s to import [I 2023-09-08 14:15:04.247 SingleUserLabApp utils:78] Package jupyter\_server\_mathjax took 0.0018s to import [I 2023-09-08 14:15:04.252 SingleUserLabApp utils:78] Package jupyter\_server\_terminals took 0.0041s to import [I 2023-09-08 14:15:04.360 SingleUserLabApp utils:78] Package jupyter\_server\_ydoc took 0.1075s to import [I 2023-09-08 14:15:04.361 SingleUserLabApp utils:78] Package jupyterlab took 0.0000s to import [I 2023-09-08 14:15:04.376 SingleUserLabApp utils:78] Package jupyterlab\_git took 0.0151s to import [I 2023-09-08 14:15:04.378 SingleUserLabApp utils:78] Package jupyterlab\_templates took 0.0014s to import [W 2023-09-08 14:15:04.378 SingleUserLabApp utils:91] A `_jupyter_server_extension_points` function was not found in jupyterlab\_templates. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server. [I 2023-09-08 14:15:04.379 SingleUserLabApp utils:78] Package nbclassic took 0.0000s to import [W 2023-09-08 14:15:04.381 SingleUserLabApp utils:91] A `_jupyter_server_extension_points` function was not found in nbclassic. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server. [I 2023-09-08 14:15:04.381 SingleUserLabApp utils:78] Package nbdime took 0.0000s to import [I 2023-09-08 14:15:04.382 SingleUserLabApp utils:78] Package notebook\_shim took 0.0000s to import [W 2023-09-08 14:15:04.382 SingleUserLabApp utils:91] A `_jupyter_server_extension_points` function was not found in notebook\_shim. Instead, a `_jupyter_server_extension_paths` function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server. [W 2023-09-08 14:15:04.387 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [W 2023-09-08 14:15:04.388 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [I 2023-09-08 14:15:04.388 SingleUserLabApp manager:344] xxx\_extension | extension was successfully linked. [I 2023-09-08 14:15:04.389 SingleUserLabApp manager:344] jupyter\_env\_whitelist | extension was successfully linked. [W 2023-09-08 14:15:04.393 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [W 2023-09-08 14:15:04.394 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [I 2023-09-08 14:15:04.395 SingleUserLabApp manager:344] jupyter\_server\_fileid | extension was successfully linked. [W 2023-09-08 14:15:04.408 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [W 2023-09-08 14:15:04.409 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [I 2023-09-08 14:15:04.409 SingleUserLabApp manager:344] jupyter\_server\_mathjax | extension was successfully linked. [W 2023-09-08 14:15:04.414 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [W 2023-09-08 14:15:04.415 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [I 2023-09-08 14:15:04.415 SingleUserLabApp manager:344] jupyter\_server\_terminals | extension was successfully linked. [W 2023-09-08 14:15:04.420 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [W 2023-09-08 14:15:04.421 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [I 2023-09-08 14:15:04.421 SingleUserLabApp manager:344] jupyter\_server\_ydoc | extension was successfully linked. [W 2023-09-08 14:15:04.427 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [I 2023-09-08 14:15:04.427 SingleUserLabApp manager:344] jupyterlab | extension was successfully linked. [I 2023-09-08 14:15:04.427 SingleUserLabApp manager:344] jupyterlab\_git | extension was successfully linked. [I 2023-09-08 14:15:04.428 SingleUserLabApp manager:344] jupyterlab\_templates | extension was successfully linked. [W 2023-09-08 14:15:04.430 NotebookApp] ‘ip’ has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. [W 2023-09-08 14:15:04.431 NotebookApp] ‘ip’ has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. [W 2023-09-08 14:15:04.431 NotebookApp] ‘ip’ has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. [W 2023-09-08 14:15:04.434 SingleUserLabApp configurable:200] Config option `open_browser` not recognized by `SingleUserLabApp`. Did you mean `browser`? [I 2023-09-08 14:15:04.434 SingleUserLabApp manager:344] nbclassic | extension was successfully linked. [I 2023-09-08 14:15:04.434 SingleUserLabApp manager:344] nbdime | extension was successfully linked. /opt/conda/lib/python3.10/site-packages/IPython/paths.py:62: UserWarning: IPython dir ‘/home/jovyan/.ipython’ is not a writable location, using a temp directory. warn(“IPython dir ‘{0}’ is not a writable location,” [I 2023-09-08 14:15:04.441 SingleUserLabApp utils:78] Package jupyterlab\_templates.extension took 0.0000s to import [I 2023-09-08 14:15:04.441 SingleUserLabApp nbserver:127] jupyterlab\_templates.extension | extension was found and enabled by notebook\_shim. Consider moving the extension to Jupyter Server’s extension paths. [I 2023-09-08 14:15:04.441 SingleUserLabApp manager:344] jupyterlab\_templates.extension | extension was successfully linked. [I 2023-09-08 14:15:04.441 SingleUserLabApp manager:344] notebook\_shim | extension was successfully linked. [W 2023-09-08 14:15:04.443 SingleUserLabApp serverapp:1904] Customizing authentication via ServerApp.login\_handler\_class=\ is deprecated in Jupyter Server 2.0. Use ServerApp.identity\_provider\_class. Falling back on legacy authentication. [I 2023-09-08 14:15:04.476 SingleUserLabApp manager:364] notebook\_shim | extension was successfully loaded. [I 2023-09-08 14:15:04.477 SingleUserLabApp manager:364] xxx\_extension | extension was successfully loaded. [I 2023-09-08 14:15:04.477 SingleUserLabApp manager:364] jupyter\_env\_whitelist | extension was successfully loaded. [I 2023-09-08 14:15:04.478 FileIdExtension] Configured File ID manager: ArbitraryFileIdManager [I 2023-09-08 14:15:04.478 FileIdExtension] ArbitraryFileIdManager : Configured root dir: /home/jovyan/notebook [I 2023-09-08 14:15:04.478 FileIdExtension] ArbitraryFileIdManager : Configured database path: /home/jovyan/.local/share/jupyter/file\_id\_manager.db [I 2023-09-08 14:15:04.479 FileIdExtension] ArbitraryFileIdManager : Successfully connected to database file. [I 2023-09-08 14:15:04.479 FileIdExtension] ArbitraryFileIdManager : Creating File ID tables and indices with journal\_mode = DELETE [I 2023-09-08 14:15:04.508 FileIdExtension] Attached event listeners. [I 2023-09-08 14:15:04.508 SingleUserLabApp manager:364] jupyter\_server\_fileid | extension was successfully loaded. [I 2023-09-08 14:15:04.509 SingleUserLabApp manager:364] jupyter\_server\_mathjax | extension was successfully loaded. [I 2023-09-08 14:15:04.514 SingleUserLabApp manager:364] jupyter\_server\_terminals | extension was successfully loaded. [I 2023-09-08 14:15:04.515 SingleUserLabApp manager:364] jupyter\_server\_ydoc | extension was successfully loaded. [I 2023-09-08 14:15:04.516 LabApp] JupyterLab extension loaded from /opt/conda/lib/python3.10/site-packages/jupyterlab [I 2023-09-08 14:15:04.517 LabApp] JupyterLab application directory is /opt/conda/share/jupyter/lab [I 2023-09-08 14:15:04.521 SingleUserLabApp manager:364] jupyterlab | extension was successfully loaded. [I 2023-09-08 14:15:04.527 SingleUserLabApp manager:364] jupyterlab\_git | extension was successfully loaded. [I 2023-09-08 14:15:04.528 SingleUserLabApp extension:134] Installing jupyterlab\_templates handler on path /user/xxx/templates [I 2023-09-08 14:15:04.528 SingleUserLabApp extension:148] Search paths: /opt/conda/lib/python3.10/site-packages/jupyterlab\_templates/templates /home/jovyan/.local/share/jupyter/notebook\_templates /opt/conda/share/jupyter/notebook\_templates /usr/local/share/jupyter/notebook\_templates /usr/share/jupyter/notebook\_templates [I 2023-09-08 14:15:04.529 SingleUserLabApp extension:151] Available templates: [I 2023-09-08 14:15:04.529 SingleUserLabApp manager:364] jupyterlab\_templates | extension was successfully loaded. [I 2023-09-08 14:15:04.530 SingleUserLabApp extension:134] Installing jupyterlab\_templates handler on path /user/xxx/templates [I 2023-09-08 14:15:04.530 SingleUserLabApp extension:148] Search paths: /opt/conda/lib/python3.10/site-packages/jupyterlab\_templates/templates /home/jovyan/.local/share/jupyter/notebook\_templates /opt/conda/share/jupyter/notebook\_templates /usr/local/share/jupyter/notebook\_templates /usr/share/jupyter/notebook\_templates [I 2023-09-08 14:15:04.531 SingleUserLabApp extension:151] Available templates: [I 2023-09-08 14:15:04.531 SingleUserLabApp manager:364] jupyterlab\_templates.extension | extension was successfully loaded. [I 2023-09-08 14:15:04.538 SingleUserLabApp manager:364] nbclassic | extension was successfully loaded. [I 2023-09-08 14:15:04.732 SingleUserLabApp manager:364] nbdime | extension was successfully loaded. * * * | | | |\_ \_\_ **| |** _| |_ \_\_\_ | |_| | ’_ / _` / _` | / -) \_ **/| .** /\ __,_\__ ,_|\__\_\_\_| |\_| Read the migration plan to Notebook 7 to learn about the new features and the actions to take if you are using extensions. [https://jupyter-notebook.readthedocs.io/en/latest/migrate\_to\_notebook7.html](https://jupyter-notebook.readthedocs.io/en/latest/migrate_to_notebook7.html) Please note that updating to Notebook 7 might break some of your extensions. [I 2023-09-08 14:15:04.735 SingleUserLabApp mixins:670] Starting jupyterhub-singleuser server version 3.1.1 [W 2023-09-08 14:15:04.753 SingleUserLabApp \_version:68] jupyterhub version 3.0.0 != jupyterhub-singleuser version 3.1.1. This could cause failure to authenticate and result in redirect loops! [I 2023-09-08 14:15:04.754 SingleUserLabApp serverapp:2792] Serving notebooks from local directory: /home/jovyan/notebook [I 2023-09-08 14:15:04.754 SingleUserLabApp serverapp:2792] Jupyter Server 2.5.0 is running at: [I 2023-09-08 14:15:04.754 SingleUserLabApp serverapp:2792] http//jupyter-xxx:8888/user/xxx/lab?token=xxxxxxxxxxxx [I 2023-09-08 14:15:04.754 SingleUserLabApp serverapp:2792] http//127.0.0.1:8888/user/xxx/lab?token=xxxxxxxxxxxx [I 2023-09-08 14:15:04.754 SingleUserLabApp serverapp:2793] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). [C 2023-09-08 14:15:04.765 SingleUserLabApp serverapp:2856] ``` To access the server, open this file in a browser: file:///home/jovyan/.local/share/jupyter/runtime/jpserver-7-open.html Or copy and paste one of these URLs: http//jupyter-xxx:8888/user/xxx/lab?token=xxxxxxxxxxxx http//127.0.0.1:8888/user/xxx/lab?token=xxxxxxxxxxxx ``` [I 2023-09-08 14:15:04.766 SingleUserLabApp mixins:591] Updating Hub with activity every 300 seconds [I 2023-09-08 14:15:19.928 SingleUserLabApp log:186] 302 GET /user/xxx/ → /user/xxx/lab? (@100.65.56.192) 0.89ms Used couple of extension that causes pod to take time to start? Any suggestion to start a user server without wait time. Thanks, Srujan ---

**Author:** ![manics](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/manics/32/85_2.png) [@manics](https://discourse.jupyter.org/u/manics)\ **Post date:** [September 11, 2023, 1:04pm UTC](https://discourse.jupyter.org/t/jupyterhub-single-user-sever-not-starting-immediately-on-kubernetes-cluster/21304/4 "2023-09-11T13:04:21Z")

Can you try obtaining the _timestamps_ for the kubernetes events, the hub logs, and the singleuser pod logs? This will show exactly where the delay is occurring. ---

**Author:** ![srujan\_gajula](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/srujan_gajula/32/10958_2.png) [@srujan\_gajula](https://discourse.jupyter.org/u/srujan_gajula)\ **Post date:** [September 13, 2023, 3:04pm UTC](https://discourse.jupyter.org/t/jupyterhub-single-user-sever-not-starting-immediately-on-kubernetes-cluster/21304/6 "2023-09-13T15:04:08Z")

Thank you for response @manics . I’ve observed this in user server pod logs [I 2023-09-08 14:15:\*\*04\*\*.766 SingleUserLabApp mixins:591] Updating Hub with activity every 300 seconds [I 2023-09-08 14:15:\*\*19.\*\*928 SingleUserLabApp log:186] 302 GET /user/xxx/ → /user/xxx/lab? (@100.65.56.192) 0.89ms In above logs first and second log has 15sec gap, for redirect the url, can we reduce that time frame?