CPU limits per user on TLJH

JupyterHub The Littlest JupyterHub
1

We’ve setup TLJH for an upcoming course, and the tutors are reporting that servers they’re testing with parallel code are using all available cores on the host, rather than the 2 that are configured for a single user.

Is there any way of enforcing this within TLJH?

2

TLJH uses SystemdSpawner where you can enforce limits on resources using systemd. Here are the config details. Not sure what is the best way to do it on TLJH though!

2 Likes
3

From the docs, these should set the systemd limits:

sudo tljh-config set limits.memory 4G
sudo tljh-config set limits.cpu 2

Are you seeing that these limits are set but not enforced?

4

Yes, I had set these values, and CPU_LIMIT appears to be set correctly in the shell.
I asked because one of the tutors for this course reported CPU usage exceeding this limit, but perhaps that wasn’t the case.

1 Like
5

ok, it should be set with those. You should be able to test if this is the case with a sample task like this:

import time
from concurrent.futures import ProcessPoolExecutor

import psutil


def use_cpu(seconds: float | int):
    """busywait to use the cpu in a single thread for a number of seconds"""
    start = time.perf_counter()
    deadline = start + seconds
    while time.perf_counter() < deadline:
        x = sum(range(100_000))


p = psutil.Process()
n_cpus = 4
with ProcessPoolExecutor(n_cpus) as pool:
    # submit tasks to use all cpus for 2 seconds
    for i in range(n_cpus):
        f = pool.submit(use_cpu, seconds=2)

    # measure cpu time of all children for one second
    children = p.children()
    before_times = sum(sum(_p.cpu_times()[:2]) for _p in children)
    time.sleep(1)
    after_times = sum(sum(_p.cpu_times()[:2]) for _p in children)

cpu_usage = after_times - before_times

where cpu_usage will be the number of cpus used. If you’ve set the limit to 2, you should get a result around 2, even if you run it with 4 or 10 processes. If you get a number that’s approximately n_cpus even when it’s larger than the limit, that meanst he limit is not having the desired effect.

1 Like