Podman(1) not working inside JupyterLab containers that are launched by JupyterHub

General
, , ,
7

UPDATE2:

Okay, the following appears to be a material difference in the docker inspect jupyter-janedoe (JSON) output between both cases, which may be contributing to this issue:

docker inspect jupyter-janedoe (launched via CLI):

"HostConfig": {
    "Binds": [
          "/sys/fs/cgroup:/sys/fs/cgroup:ro"
 ],

docker inspect jupyter-janedoe (launched via DockerSpawner):

"HostConfig": {
    "Binds": [
          "jupyterhub-user-janedoe:/home/jovyan:rw"
 ],

The DockerSpawner entry should be a union of both entries (as shown below), but it’s missing the second item:

"HostConfig": {
    "Binds": [
          "jupyterhub-user-janedoe:/home/jovyan:rw",
          "/sys/fs/cgroup:/sys/fs/cgroup:ro"
 ],

This means the code-snippet in my UPDATE1 is incorrect.

So far I’ve verified that updating the c.DockerSpawner.volumes.update attribute is incorrect, because it led to the following at runtime, which isn’t needed (and maybe even ambiguous with respect to what it even means LoL :blush:):

# We want a Volume entry, not Device entry.
"Devices": [{
    "PathOnHost": "/sys/fs/cgroup",
    "PathInContainer": "/sys/fs/cgroup",
    "CgroupPermissions": "ro"
}]

So reverse-engineering a little bit, according to the CLI working case, we want the following /sys/fs/cgroup entries; but only the "Volumes": (middle) entry appears in the DockerSpawner failure case (the outer two entries are missing):

[ ... snip ... ]

"Mounts": [{
   "Type": "bind",
   "Source": "/sys/fs/cgroup",
   "Destination": "/sys/fs/cgroup",
   "Mode": "ro",
   "RW": false,
   "Propagation": "rprivate" }]

[ ... snip ... ]

"Volumes": {
   "/sys/fs/cgroup": {}
}

[ ... snip ... ]

"HostConfig": {
   "Binds": [
     "/sys/fs/cgroup:/sys/fs/cgroup:ro"
],

To achieve this, I’m trying to figure out what the correct attribute(s) is/are to modify, as well as the correct syntax.

In summary, I need a Mounts: entry for /sys/fs/cgroup; as well as to append /sys/fs/cgroup entry to the HostConfig list.