Limit Resources and Idle Culler

JupyterHub
1

Hello! Newbie here…
I installed a not so powerfull Oracle Linux 9 VM with the available resources: 12GB RAM and 2 vCPUs. It is used in classes in a satisfactory way. With more than 25 users, things sometimes go bad.
I already configured c.JupyterHub.shutdown_on_logout = True and this helps.
I also set up Idle Culler, but it is not working, but the process is running:

in jupyterhub_config.py

import sys
c.JupyterHub.services = [
{
“name”: “idle-culler”,
“command”: [
sys.executable, “-m”,
“jupyterhub_idle_culler”,
“–timeout=1200”
],
}
]

c.JupyterHub.load_roles = [
{
“name”: “idle-culler”,
“description”: “Culls idle servers”,
“scopes”: [“read:users:name”, “read:users:activity”, “servers”],
“services”: [“idle-culler”],
}
]

what am I missing?
Also, is there a way to limit user CPU and memory? Sometimes a loop made by a a student is enough to kill the server, sometimes it is not.

Versions:
Selected Jupyter core packages…
IPython : 8.18.1
ipykernel : 6.29.5
ipywidgets : 8.1.5
jupyter_client : 8.6.3
jupyter_core : 5.7.2
jupyter_server : 2.14.2
jupyterlab : 4.2.5
nbclient : 0.10.0
nbconvert : 7.16.4
nbformat : 5.10.4
notebook : 7.2.2
qtconsole : not installed
traitlets : 5.14.3

system status jupyterhub:
/opt/jupyterhub/bin/python3 /opt/jupyterhub/bin/jupyterhub -f /opt/jupyterhub/etc/jupyterhub/jupyterhub_config.py
node /usr/local/bin/configurable-http-proxy --ip “” --port 8000 --api-ip 127.0.0.1 --api-port 8001 --error-target http://127.0.0.1:8081/hub/error --log-level info --ssl-key /opt/jupyterhub/etc/certificado/>
/opt/jupyterhub/bin/python3 -m jupyterhub_idle_culler --timeout=1200

Thanks and regards.
Dave

2

Also, is there a way to limit user CPU and memory?

Maybe, it depends on what spawner is used. What spawner is jupyterhub configured to use?

(Not sure about why culling fail, bug a full jupyterhub config and logs from jupyterhub itself are relevant to debug it)

3

Thanks for the reply.
I guess the spawner is the way each notebook is started. if yes, then:
/opt/jupyterhub/bin/python3 -m ipykernel_launcher -f /home/students/xxxx

By the way…any chance an admin can stop other user services from cmd?
“jupyter notebook stop port_number” stops user’s own server.
Thanks.

4

Hello again. Still could not find a way do cull idle servers.
I also tried the following as in the official documentation:

shutdown the server after no activity for an hour
c.ServerApp.shutdown_no_activity_timeout = 40 * 60

shutdown kernels after no activity for 10 minutes
c.MappingKernelManager.cull_idle_timeout = 10 * 60

Check every minute
c.MappingKernelManager.cull_interval = 60

cull connected (e.g. Browser tab open)
c.MappingKernelManager.cull_connected = True

This was supposed to allow better control, but I can still see running servers after the timeouts. The logs don’t show anything “stop”/“cull” related. What am I missing? Thanks!

5

Although there are still some fine tuning to do, I am now able to stop idle notebooks, either with or without open browser tabs.
It seems to be working better with Edge browser, but I’ll keep investigating. What I have done was:
c.JupyterHub.spawner_class = ‘jupyterhub.spawner.LocalProcessSpawner’
c.Spawner.args = [
‘–NotebookApp.shutdown_no_activity_timeout=600’,
‘–MappingKernelManager.cull_idle_timeout=600’,
‘–MappingKernelManager.cull_interval=60’,
‘–MappingKernelManager.cull_connected=True’,
]
So basically, send the arguments to each individual spawner. I can then confirm that the process runs with these parameters, unlike when they were defined like in my previous post.