When I click `add new service`, instead of finding the local image, jupyterhub goes to the image warehouse to pull a new image


Problem description:
The image pull strategy is c.KubeSpawner.image_pull_policy = 'IfNotPresent'. When I add a new service many times, some images are obtained locally (successfully added) and some images are obtained from the image warehouse (failed to add). What caused this problem?

Question reference:
According to k8s’s image pull policy, when the policy is IfNotPresent, program can only go to the image warehouse to obtain the image if there is no image locally or the image’s tag is latest

Envision solutions:
I expect that when a local image exists, the jupyterhub acquires the image locally

Problem screenshot:

My deployment environment information:

  • jupyterhub environment information
    jupyterhub                         1.4.2
    jupyterhub-kubespawner             1.1.0
    jupyterhub-dummyauthenticator      0.3.1
  • Jupyterhub brief configuration information
    c.KubeSpawner.image = 'jupyterhub/singleuser:1.0.0'
    c.KubeSpawner.cmd = ["jupyter-labhub","--notebook-dir=/home/jovyan/work/"]
    c.KubeSpawner.image_pull_policy = 'IfNotPresent'
    c.KubeSpawner.namespace = 'jupyterhub-k8s'
  • docker images information
    jupyterhub/singleuser                   1.0.0               9fa09c6615a5        8 months ago        651MB
    jupyterhub/singleuser                   latest              9fa09c6615a5        8 months ago        651MB


Can you see kubectl describe pod jupyter-test2 to see the events related to the pod, and make sure that the imagePullPolicy was indeed set? And can you verify that the image was indeed available on each node when the pod was started?

What cloud provider / kubernetes deployment are you using? I ran into a lot of frustration with e.g. OVHCloud which silently and unconditionally forces imagePullPolicy to be Always, no matter what configuration you use when you create the pod.

1 Like

@minrk Thanks!

As you said, not all node nodes in my k8s environment have images