Connecting to Kernel from outside Docker image

Hello,
I’ve been trying to start a Jupyter Kernel inside of a jupyter/datascience-notebook Docker container, and connect to it from outside the container with the emacs-jupyter package. I tried to override and forward the relevant ports like this:

docker run -p 8888:8888 \
           -p 56406:56406 \
           -p 56407:56407 \
           -p 56408:56408 \
           -p 56409:56409 \
           -p 56410:56410 \
           jupyter/datascience-notebook \
           jupyter-kernel --ip=0.0.0.0 --NotebookApp.token='' \
           --KernelManager.control_port=56406 \
           --KernelManager.hb_port=56407 \
           --KernelManager.iopub_port=56408 \
           --KernelManager.shell_port=56409 \
           --KernelManager.stdin_port=56410 \
           --KernelManager.connection_file=~/notebooks/.jupyter_confile.json

When I specified the KernelManager.show_config_json property, the overriden ports were reflected in the output.

$ docker run \
    -v .:/home/jovyan/notebooks \
    -p 8888:8888 \
    -p 56406:56406 \
    -p 56407:56407 \
    -p 56408:56408 \
    -p 56409:56409 \
    -p 56410:56410 \
    jupyter/datascience-notebook \
    jupyter-kernel --ip=0.0.0.0 --NotebookApp.token='' \
    --KernelManager.control_port=56406 \
    --KernelManager.hb_port=56407 \
    --KernelManager.iopub_port=56408 \
    --KernelManager.shell_port=56409 \
    --KernelManager.stdin_port=56410 \
    --KernelManager.connection_file=~/notebooks/.jupyter_confile.json \
    --Application.show_config_json=True
{
 "Application": {},
 "KernelManager": {
  "connection_file": "/home/jovyan/notebooks/.jupyter_confile.json",
  "control_port": 56406,
  "hb_port": 56407,
  "iopub_port": 56408,
  "ip": "0.0.0.0",
  "shell_port": 56409,
  "stdin_port": 56410
 },
 "NotebookApp": {
  "token": ""
 }
}

However, when I look at the actual connection file created by Jupyter, I still saw randomized ports like this:

{
  "shell_port": 57421,
  "iopub_port": 42961,
  "stdin_port": 58875,
  "control_port": 50919,
  "hb_port": 50977,
  "ip": "0.0.0.0",
  "key": "<omitted incase it's private>",
  "transport": "tcp",
  "signature_scheme": "hmac-sha256",
  "kernel_name": ""
}

In case this was an issue with just the Docker image, I tried to replicate it outside of Docker, by running just:

$ jupyter-kernel --ip=0.0.0.0 --NotebookApp.token='' \
    --KernelManager.control_port=56406 \
    --KernelManager.hb_port=56407 \
    --KernelManager.iopub_port=56408 \
    --KernelManager.shell_port=56409 \
    --KernelManager.stdin_port=56410 \
    --KernelManager.connection_file=/tmp/confile.json \
    --Application.show_config_json=True
{
 "Application": {},
 "KernelManager": {
  "connection_file": "/home/sridaran/notebooks/.jupyter_confile.json",
  "control_port": 56406,
  "hb_port": 56407,
  "iopub_port": 56408,
  "ip": "0.0.0.0",
  "shell_port": 56409,
  "stdin_port": 56410
 },
 "NotebookApp": {
  "token": ""
 }
}

When I ran the above command without the show_config_json parameter, I found that the ports were once again being randomized.

$ cat /tmp/confile.json
{
  "shell_port": 48209,
  "iopub_port": 34035,
  "stdin_port": 43771,
  "control_port": 46785,
  "hb_port": 48043,
  "ip": "0.0.0.0",
  "key": "<omitted incase it's private>",
  "transport": "tcp",
  "signature_scheme": "hmac-sha256",
  "kernel_name": ""
}

Can someone please explain what I’m doing wrong and how to fix it? I believe the reason I haven’t been able to get my setup working is because I am not forwarding the ports that are actually being used by Jupyter from my container.

For reference, I took inspiration from this GitHub issue, but there’s a chance that I misinterpreted something.