DockerSpawner emits: [[**Spawn failed: The 'ip' trait of a Server instance expected a unicode string, not the NoneType None.]] on a docker image that is cloned with zero changes (via docker(1) commit ...)

JupyterHub
, ,
1

Hello Friends:

I have several custom docker images that successfully launch under JupyterHub's DockerSpawner.

However, when I launch any of those images like this:

root# docker run -it --rm --privileged -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
      acme/acme-notebook:1.0 /bin/bash

and then literally make zero changes inside it (meaning I issue absolutely no commands while in that session), and then finally in another terminal commit it:

root# docker commit <above-container-id> acme/acme-notebook:1.1

When that committed image is selected via the JupyterHub dropdown menu, the following error occurs:

Error: HTTP 500: Internal Server Error (Error in Authenticator.pre_spawn_start:
TraitError The 'ip' trait of a Server instance expected a unicode string, not
the NoneType None.)

Yet no changes were made, commands wise.

I get the same failure when trivially committing the Jupyter Stacks image:

  • jupyter/all-spark-notebook:latest

Because certain build-actions cannot be performed at docker(1) build [...] time, it is necessary to modify and commit images after the build, but I’m having this issue when I do (even when that change is a null change).

This exception keeps reoccurring on trivial things; for example here too (where inclusion of a tiny spawn hook script causes the same exception).

My JupyterHub is something I also built from scratch as a Docker, container and runs well. Here’s is it’s configuration as of last year; but I can post it again (it’s changed a little bit). Perhaps there’s a trait / setting that will reveal something.

This looks like a bug, but I don’t know where to even start to debug this (the same issue that keeps arising in various scenarios).

Thank you.

2

When you run a container you’re making changes just by the fact that you’ve run it.

In this case you’ve changed the Cmd, as you can see by inspecting the images where tmp is the committed image:

$ docker image inspect jupyter/minimal-notebook:latest --format '{{.Config.Cmd}}'
[start-notebook.sh]

$ docker image inspect tmp --format '{{.Config.Cmd}}'
[/bin/bash]
1 Like
3

Excellent! I hadn’t realized that, but it makes sense.

I suppose I can make necessary changes to a live JupyterLab container (launched by JupyterHub) via docker exec [...] into it, and then commit that.

(But I thought I tried that – it’s bee a long two days. :sweat:)

I’ll try the above, but just in case, is there a way to edit an image to change that back? Kind of off-topic I know. :slight_smile:

Thank you.

4

Hi @manics

Performing a docker commit [...] of a JupyterHub launched & running JupyerLab container, when later launched winds up with this error:

Spawn failed: Server at http://172.10.0.4:8888/user/jdoe/ 
didn't respond in 30 seconds

And that’s with the following settings in ./jupyterhub_config.py to increase launch-time timeout (and where one line is commented out below, shown for clarity):

c.DockerSpawner.start_timeout = 300
#c.DockerSpawner.http_timeout = 120
5

The documentation for docker commit has several options including for setting CMD, maybe that will help?

6

@manics

Thanks for drawing my attention to this. I thought all of the options below were trivial. I’ll try --change options and see. If you want and have a moment, see if committing a running Lab does the same on your end (thank you).

jdoe@fedora33$ docker commit --help

Usage:  docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]

Create a new image from a container's changes

Options:
  -a, --author string    Author (e.g., "John Hannibal Smith <hannibal@a-team.com>")
  -c, --change list      Apply Dockerfile instruction to the created image
  -m, --message string   Commit message
  -p, --pause            Pause container during commit (default true)
jdoe@fedora33$ docker commit --change --help
"docker commit" requires at least 1 and at most 2 arguments.
See 'docker commit --help'.

Usage:  docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]