Binderhub with private docker registry and token_url

Binder
1

Hi!

I’m running into an issue with Docker pull authentication (Error: ErrImagePull) when deploying BinderHub using the following secrets.yaml passed to Helm (version 1.0.0-0.dev.git.3724.h837169df):

registry:
  url: "https://binder-registry.conp.cloud"
  username: xxx
  password: yyy

This setup previously worked when using use_registry: true and setting image_prefix: a/b/c-, as long as the Docker credentials were also present in .docker/config.json on every node (via docker login).

However, Binder can no longer pull images from our private registry with these givens.

When I add the following to the configuration:

  DockerRegistry:
    token_url: None

the behavior changes. Instead of failing with ErrImagePull, Binder tries to build a new image (even if it already exists in the registry). It successfully pushes the rebuilt image to the private registry, but subsequent launches still trigger a fresh build each time.

Is native basic HTTP authentication no longer supported? If so, it seems I’ll need to add an authentication middleware to our registry, which is doable, but I’d rather stick with a faster solution if available.

Thanks in advance for any advice!

2

Dear @agahkarakuzu,

without knowing what container registry you are using we can only make a guest what is causing the problem.

In earlier March, we noticed that BinderHub were having some problems to connect with some container registries. This was fixed in Append / to Docker Hub auth URL by rgaiacs · Pull Request #1945 · jupyterhub/binderhub · GitHub.

3

Dear @rgaiacs thanks for your response.

Sorry it was not clear in my first comment, we are using a self-hosted private docker registry. Deployed in docker swarm mode with traefik as reverse proxy, if those details are relevant.

Currently, we don’t have an authentication layer on the registry side, it only provides basic HTTP authentication, and I was wondering if this is no longer an expected case.

On the other hand, binder can still push images to this private registry, so basic auth should not be a complete roadblock.

4

I just solved this problem by simply adding imagePullSecrets under hub:extraConfig as in:

kubectl create secret docker-registry userpull \
  --docker-server=REGISTRY_URL \
  --docker-username=USERNAME \
  --docker-password=PASSWORD \
  --namespace=binderhub

then

hub:
  extraConfig:
    imagePullSecrets: |
      c.KubeSpawner.image_pull_secrets = ['userpull']

This actually has more to do with hub than binder. Such extra configuration is needed probably because we are running on OpenStack and using our own registry, but still could be useful if anyone runs into a similar problem.

2 Likes