Troubleshoot - terminal hangs on launch, Docker image fails in Linux, works in macOS

JupyterLab
1

Greetings.

A terminal launched in a Jupyter Lab hangs on a blank screen indefinitely when attempting to launch in an official Docker image hosted in Ubuntu; the same Docker image works without issues under macOS. Same image, same docker-compose.yaml file in both cases.

I need help understanding how this works in Jupyter, required port connections, configuration, and where to look for an exception log in the Jupyter environment. This may be a bug in the jupyter/datascience-notebook image, in which case I’ll be happy to help resolve it (or define a good workaround) – I don’t know enough about the Jupyter environment to troubleshoot this. Thanks in advance and have a great day!

Environment

Image: jupyter/datascience-notebook:latest
Docker ver. 19.03.5 under both macOS and Ubuntu Server 16.04 LTS, Ubuntu Server 18.04 LTS

Console output

alian.cime.dev        | [I 18:22:45.064 NotebookApp] Kernel started: 0f20d781-1d9f-4331-bfc0-ac38c59b93ff
alian.cime.dev        | [I 18:22:45.066 NotebookApp] Build is up to date
alian.cime.dev        | [I 18:22:55.262 NotebookApp] New terminal with automatic name: 1
alian.cime.dev        | [W 18:23:19.143 NotebookApp] 400 GET /api/kernels/0f20d781-1d9f-4331-bfc0-ac38c59b93ff/channels?session_id=d5b2ffbd-f93b-4106-b998-10e36bcc3d98 (172.18.0.1) 11.48ms referer=None
alian.cime.dev        | [W 18:24:19.268 NotebookApp] 400 GET /api/kernels/0f20d781-1d9f-4331-bfc0-ac38c59b93ff/channels?session_id=3cd353ee-6604-44c6-b5b0-0bc8ae160510 (172.18.0.1) 3.76ms referer=None
alian.cime.dev        | [W 18:25:19.313 NotebookApp] 400 GET /terminals/websocket/1 (172.18.0.1) 1.26ms referer=None
alian.cime.dev        | [W 18:26:19.420 NotebookApp] Replacing stale connection: 0f20d781-1d9f-4331-bfc0-ac38c59b93ff:d5b2ffbd-f93b-4106-b998-10e36bcc3d98
alian.cime.dev        | [W 18:27:39.381 NotebookApp] 400 GET /api/kernels/0f20d781-1d9f-4331-bfc0-ac38c59b93ff/channels?session_id=501687fc-ce6d-4904-9bc8-a2d22c2f34e0 (172.18.0.1) 3.29ms referer=None
alian.cime.dev        | [W 18:28:39.445 NotebookApp] Replacing stale connection: 0f20d781-1d9f-4331-bfc0-ac38c59b93ff:3cd353ee-6604-44c6-b5b0-0bc8ae160510
alian.cime.dev        | [W 18:29:59.462 NotebookApp] 400 GET /terminals/websocket/1 (172.18.0.1) 1.09ms referer=None
alian.cime.dev        | [W 18:30:59.524 NotebookApp] Replacing stale connection: 0f20d781-1d9f-4331-bfc0-ac38c59b93ff:d5b2ffbd-f93b-4106-b998-10e36bcc3d98
alian.cime.dev        | [W 18:32:19.550 NotebookApp] Replacing stale connection: 0f20d781-1d9f-4331-bfc0-ac38c59b93ff:501687fc-ce6d-4904-9bc8-a2d22c2f34e0
alian.cime.dev        | [W 18:33:39.581 NotebookApp] Replacing stale connection: 0f20d781-1d9f-4331-bfc0-ac38c59b93ff:3cd353ee-6604-44c6-b5b0-0bc8ae160510
1 Like
2

Update: the issue is poor websocket configuration in the Jupyter stack and how it handles connections when using a reverse proxy (necessary for HTTPS termination); there’s some information from this bug report. Now that I know the issue I’ll try to figure a good configuration set up, perhaps share it with the group.

Cheers!

1 Like
3

Hi, were you able to find a solution for this issue?

4

I have resolved the issue. It is actually two folds/steps process:

  1. Configure NGINX proxy

  2. Setting c.NotebookApp.allow_origin = '*'

  3. For server directive that is listening on 443 ssl:

location / {

     proxy_set_header        Host $host;
     proxy_set_header        X-Real-IP $remote_addr;
     proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header        X-Forwarded-Proto $scheme;

     set $upstream_app jupyter-tensorflow-notebook;
     set $upstream_port 8888;
     set $upstream_proto http;
     proxy_pass $upstream_proto://$upstream_app:$upstream_port;
     proxy_read_timeout  90;

     # WebSocket support
     proxy_http_version 1.1;
     proxy_set_header Upgrade $http_upgrade;
     proxy_set_header Connection "upgrade";
 }
  1. Then update jupyter_notebook_config.py and set c.NotebookApp.allow_origin = ‘*’ there.