Usage of the `singleuser.image.name` configuration

[hiroyuki-sato]

Hello, members.

Does someone tell me how to how to specify a private image registry?
(Usage of the singleuser.image.name configuration)

1. Environment

• Z2JH: 3.0.3
• k8s: v1.27.4
• OS: Ubuntu 22.04

2. Question

• Could you tell me how to specify a private image registry?
  (It it possible specify http instead of https?)
• Could you tell me how to remove stacked image?

2.1. Setup configuration

singleuser:
  storage:
    capacity: 100Mi
    homeMountPath: /home/jovyan
    dynamic:
      pvcNameTemplate: claim-{username}{servername}
      storageAccessModes:
      - ReadWriteMany
      storageClass: nfs-client
      volumeNameTemplate: volume-{username}{servername}
    homeMountPath: /home/jovyan
    type: dynamic
  image:
    name: '10.10.10.10:5000/jhtest'  ## How to write this configuration??
    tag: 'latest'

helm upgrade --cleanup-on-fail \
  --install jhub jupyterhub/jupyterhub \
  --namespace namespace-jupyterhub \
  --create-namespace \
  --values config.yaml

It launch hook-image-awaiter and hook-image-puller. But the status of hook-image-puller is Init:ImagePullBackOff

namespace-jupyterhub   hook-image-awaiter-z4pm9                           1/1     Running                 0              14m
namespace-jupyterhub   hook-image-puller-84fzg                            0/1     Init:ImagePullBackOff   0              14m
namespace-jupyterhub   hook-image-puller-b2p4h                            0/1     Init:ImagePullBackOff   0              14m
namespace-jupyterhub   hook-image-puller-pp7dw                            0/1     Init:ImagePullBackOff   0              14m
namespace-jupyterhub   hook-image-puller-qcb4h                            0/1     Init:ImagePullBackOff   0              14m
namespace-jupyterhub   nfs-subdir-external-provisioner-866b898dd7-x2vjv   1/1     Running                 1 (13d ago)    18d

kubectl logs -n namespace-jupyterhub hook-image-puller-qcb4h
Defaulted container "pause" out of: pause, image-pull-metadata-block (init), image-pull-singleuser (init)
Error from server (BadRequest): container "pause" in pod "hook-image-puller-qcb4h" is waiting to start: PodInitializing

2.2. Force delete pods

And also I’m trying to remove an image using do kubectl delete pods -n namespace-jupyterhub $i --grace-period=0 --force <pods-name>
It removed specified images. But a few second after, It launch new hook-image-awaiter and hook-image-puller

for i in $( kubectl get pods --all-namespaces | awk '/^namespace-jupyterhub/{ print $2 }' | grep -v nfs  ) ; do kubectl delete pods -n namespace-jupyterhub  $i --grace-period=0 --force ; done
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "hook-image-awaiter-z4pm9" force deleted
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "hook-image-puller-84fzg" force deleted
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "hook-image-puller-b2p4h" force deleted
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "hook-image-puller-pp7dw" force deleted
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "hook-image-puller-qcb4h" force deleted

It respawn new hook-image-awaiter hook-image-awaiter-kkhrm (I deleted hook-image-awaiter-z4pm9)

namespace-jupyterhub   hook-image-awaiter-kkhrm                           1/1     Running                 0              8m23s
namespace-jupyterhub   hook-image-puller-6h52v                            0/1     Init:ImagePullBackOff   0              8m32s
namespace-jupyterhub   hook-image-puller-ljk9t                            0/1     Init:ImagePullBackOff   0              8m32s
namespace-jupyterhub   hook-image-puller-rdvqg                            0/1     Init:ImagePullBackOff   0              8m31s
namespace-jupyterhub   hook-image-puller-tlzfx                            0/1     Init:ImagePullBackOff   0              8m31s
namespace-jupyterhub   nfs-subdir-external-provisioner-866b898dd7-x2vjv   1/1     Running                 1 (13d ago)    18d

3. Creating image

I setup private registry with the following steps. (I used podman istead of docker by the way)

Dockerfile

FROM python:3.11-bullseye as build-stage

COPY requirements.txt requirements.txt
ARG PIP_CACHE_DIR=/tmp/pip-cache
RUN --mount=type=cache,target=${PIP_CACHE_DIR} \
    pip install build \
 && pip wheel \
        --wheel-dir=/tmp/wheels \
        -r requirements.txt

FROM jupyterhub/k8s-singleuser-sample:3.0.3

#https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/47665a7ef7ddea9ce8e6a1cc3960917587fec660/images/singleuser-sample/Dockerfile

ENV NB_USER=jovyan \
    NB_UID=1000 \
    HOME=/home/jovyan

USER root

# install wheels built in the build-stage
COPY requirements.txt /tmp/requirements.txt
ARG PIP_CACHE_DIR=/tmp/pip-cache
RUN pip install -r /tmp/requirements.txt


WORKDIR ${HOME}
USER ${NB_USER}

EXPOSE 8888
ENTRYPOINT ["tini", "--"]
CMD ["jupyter", "lab"]

Create image

podman build  . -t jhtest

Register build image

I used registry:2 iamge.

podman run -d -p 5000:5000 --privileged -v /tmp/registry:/var/lib/registry registry:2

podman push localhost:5000/jhtest:latest --tls-verify=false

pull image docker pull from another host worked.

Best regards.

[manics]

There’s an example of using a private registry for the singleuser image in

Image Pull from Private Gitlab Registry [Access Forbidden]

Hello, I am currently facing an issue blocking me from pulling images from my Private Gitlab Container Registry. I am using JupyterHub Kubernetes deployment via Helm. Image and secret configuration: singleuser: image: name: myapp.com/repo-name/myimage tag: '0.1' imagePullSecret: email: <email> enabled: true password: <password> registry: myapp.com username: <username> Failed to pull image "myurl.com/myimage:0.1": rpc error: code = Unknown desc = Error respons…

[hiroyuki-sato]

Hello, @manics Thank you for your reply.

I set-uped SSL reverse proxy server. Then set
singleuser.image.name to host_fqdn/image_name.
Finally, z2jh connected to my private registry.

custom:
  hoge: foo
hub:
  db:
    pvc:
      storageClassName: nfs-client
debug:
  enabled: true
singleuser:
  memory:
    limit: 2G
    guarantee: 2G
  storage:
    capacity: 100Mi
    homeMountPath: /home/jovyan
    dynamic:
      pvcNameTemplate: claim-{username}{servername}
      storageAccessModes:
      - ReadWriteMany
      storageClass: nfs-client
      volumeNameTemplate: volume-{username}{servername}
    homeMountPath: /home/jovyan
    type: dynamic
  image:
    name: 'my-private-registry.fqdn/jhtest'
    tag: 'latest'

nginx reverse proxy setting.

http {
  # SNIP

    upstream registry {
        server 127.0.0.1:5000;
    }

    server {
        listen       443 ssl default_server;
        listen       [::]:443 ssl default_server;
        server_name  _;
        root         /usr/share/nginx/html;

        ssl_certificate "/path/to/cert.crt";
        ssl_certificate_key "/path/to/private.key";
        ssl_session_cache shared:SSL:1m;
        ssl_session_timeout  10m;
        ssl_ciphers PROFILE=SYSTEM;
        ssl_prefer_server_ciphers on;

        location / {
          proxy_pass http://registry;
        }

    }

}