Jupyterhub-singleuser consuming significant and constant CPU

JupyterHub
1

I’m using python 3.11 + jupyterhub 4.0.2 + jupyterlab 4.0.5 and am currently having a problem with jupyterhub-singleuser consuming constant and high CPU even when the user is not connected to the web console. I am launching it like so via jupyterhub_config.py

c.Spawner.cmd = ['bash', '-l', '-c', 'exec jupyterhub-singleuser "$@"', '--']

My reproducing steps are:

  1. Log into jupyterhub via web browser
  2. Exit jupyterhub/lab by going to File → Logout
  3. Observe on the server that jupyterhub-singleuser is running and chewing lots of CPU
akrherz  3566392 20.4  0.0 5029420 241284 ?      Ssl  10:35   1:54 /opt/miniconda3/envs/prod/bin/python3.11 /opt/miniconda3/envs/prod/bin/jupyterhub-singleuser --NotebookApp.allow_origin=jupyterhub.fqdn.redacted

Any ideas?

2

CPU usage never goes down? Are you testing this locally on your workstation using LocalProcessSpawner? Do you see anything strange in logs? Could you share them?

1 Like
3

Correct, the above now looks like this without me having any interaction with the server

akrherz  3566392 11.4  0.0 5029420 244008 ?      Ssl  10:35  16:58 /opt/miniconda3/envs/prod/bin/python3.11 /opt/miniconda3/envs/prod/bin/jupyterhub-singleuser --NotebookApp.allow_origin=...

I have not tried such a thing, this is happening on the server that my colleagues use.

Nothing logged since I logged out of the session.

4

How is your JupyterHub deployed? Even if you have logged out of JupyterLab, the backend server keeps running until you terminate the single user server. I was talking about the logs of that single user server.

5

Thank you for the response. I don’t know where the logs for the single user server are located? I only see the logs for the running of jupyterhub -f jupyterhub_config.py. Here’s my config

from oauthenticator.generic import GenericOAuthenticator

origin = 'jupyterhub...'
c.Spawner.args = [f'--NotebookApp.allow_origin={origin}']
c.JupyterHub.tornado_settings = {
    'headers': {
        'Access-Control-Allow-Origin': origin,
    },
}
c.Spawner.default_url = "/user/{username}/lab"
c.SingleUserNotebookApp.default_url = "/user/{username}/lab"
c.Spawner.ip = '127.0.0.1'
c.Spawner.cmd = ['bash', '-l', '-c', 'exec jupyterhub-singleuser "$@"', '--']

c.JupyterHub.authenticator_class = GenericOAuthenticator

c.GenericOAuthenticator.client_id = ...
c.GenericOAuthenticator.client_secret = ...

c.GenericOAuthenticator.allow_all = True
c.GenericOAuthenticator.oauth_callback_url = 'https://.../hub/oauth_callback'
c.GenericOAuthenticator.token_url = 'https://iastate.../oauth2/v1/token'
c.GenericOAuthenticator.userdata_url = 'https://iastate.../oauth2/v1/userinfo'
c.GenericOAuthenticator.username_key = "email"
c.GenericOAuthenticator.scope = ['openid', 'profile', 'email', 'address', 'phone', 'offline_access']
c.GenericOAuthenticator.login_service = "Okta@ISU"

c.NotebookApp.disable_check_xsrf = True
6

In that case your single user server logs will be logged along with JupyterHub logs. In any case, if you dont see anything strange in the logs, I assume it must be down to either polling or PeriodicCallbacks that backend jupyter_server uses. I am not quite sure though but this is my guess!

1 Like