I am trying to serve a dockerized JupyterHub behind an nginx reverse proxy on a subpath (not a subdomain).
- The reverse proxy and JupyterHub are running on the same machine.
- The reverse proxy is hosting a basic html page that links to various subpathed web applications running in docker containers on the same machine as the proxy.
- The reverse proxy terminates TLS/SSL connection from client, and the backend is all unencrypted http and ws.
- JupyterHub works perfectly fine if I connect to it directly on the exposed docker port (bypassing the proxy), but not through the proxy.
- Have tried cobbling together various nginx configuration settings from the JupyterHub docs, Stackoverflow, Reddit, and this forum.
Since JupyterHub is working fine when connecting directly to the exposed docker port, I am convinced that the issue is my nginx configuration. If someone could point me in the right direction to get it working, please let me know!
JupyterHub Version
$ docker exec jupyterhub jupyterhub --version
2.1.1
OS Version
$ cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.9 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.9"
PRETTY_NAME=RHEL
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.9
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.9"
Docker Version
$ docker version
Client: Docker Engine - Community
Version: 20.10.12
API version: 1.41
Go version: go1.16.12
Git commit: e91ed57
Built: Mon Dec 13 11:45:41 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.12
API version: 1.41 (minimum version 1.12)
Go version: go1.16.12
Git commit: 459d0df
Built: Mon Dec 13 11:44:05 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.12
GitCommit: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
runc:
Version: 1.0.2
GitCommit: v1.0.2-0-g52b36a2
docker-init:
Version: 0.19.0
GitCommit: de40ad0
JupyterHub docker-compose.yml
$ cat docker-compose.yml
version: "3.8"
services:
jupyterhub:
build: "./build"
container_name: "jupyterhub"
image: "jupyterhub_notebook"
# restart: "unless-stopped"
environment:
# - http_proxy="http://[redacted]:8080"
# - https_proxy="http://[redacted]:8080"
# - HTTP_PROXY="http://[redacted]:8080"
- HTTPS_PROXY="http://[redacted]:8080"
networks:
- "reverse_proxy"
ports:
- "15080:8000"
volumes:
- "jupyterhub_home:/home" # persist users
- "jupyterhub_etc:/etc" # persist jupyterhub_config.py, passwd, shadow
- "jupyterhub_srv_jupyterhub:/srv/jupyterhub" # persist cookie secrets
- "jupyterhub_var_spool_mail:/var/spool/mail" # persist user mail
networks:
reverse_proxy:
name: "reverse_proxy"
volumes:
jupyterhub_etc:
name: "jupyterhub_etc"
driver: "local"
driver_opts:
device: "./jupyterhub_etc"
type: "none"
o: "bind"
jupyterhub_home:
name: "jupyterhub_home"
driver: "local"
driver_opts:
device: "./jupyterhub_home"
type: "none"
o: "bind"
jupyterhub_srv_jupyterhub:
name: "jupyterhub_srv_jupyterhub"
driver: "local"
driver_opts:
device: "./jupyterhub_srv_jupyterhub"
type: "none"
o: "bind"
jupyterhub_var_spool_mail:
name: "jupyterhub_var_spool_mail"
driver: "local"
driver_opts:
device: "./jupyterhub_var_spool_mail"
type: "none"
o: "bind"
JupyterHub Dockerfile
$ cat build/Dockerfile
FROM jupyterhub/jupyterhub:latest
RUN yes | unminimize
RUN apt-get update
RUN apt-get install --assume-yes dialog
RUN apt-get install --assume-yes manpages
RUN apt-get install --assume-yes manpages-posix
RUN apt-get install --assume-yes man-db
RUN apt-get install --assume-yes apt-utils
RUN apt-get install --assume-yes --reinstall less curl
RUN apt-get install --assume-yes bash-completion man-db neovim git curl python3 python3-venv python-is-python3
RUN apt-get install --assume-yes bash-completion git
RUN apt-get install --assume-yes bash-completion neovim
RUN apt-get install --assume-yes bash-completion python3 python3-venv python-is-python3
RUN apt-get install --assume-yes sudo
RUN apt-get upgrade --assume-yes
RUN apt-get autoremove --assume-yes
RUN pip install --upgrade setuptools pip wheel
RUN pip install --upgrade jupyterlab
ENV no_proxy jupyterhub
ENV no_proxy twopyter
# Ensure env vars are sourced in Jupyter terminals
RUN echo '\n####\n# USER ADDED\n####\n' >> /etc/skel/.bashrc
RUN echo '\n#Help pip use postal proxy\nexport HTTPS_PROXY=http://[redacted]:8080\n' >> /etc/skel/.bashrc
RUN echo '\n# Add pip bin directory to user path\nexport PATH=$HOME/.local/bin:$PATH\n' >> /etc/skel/.bashrc
Nginx Version (Reverse Proxy)
$ docker exec reverse_proxy nginx -v
nginx version: nginx/1.21.6
Nginx Rules (Reverse Proxy)
$ docker exec reverse_proxy cat /etc/nginx/conf.d/default.conf
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 443 ssl;
server_name [redacted];
ssl_certificate /etc/nginx/ssl/certs/[redacted].cert.pem;
ssl_certificate_key /etc/nginx/ssl/certs/[redacted].key.pem;
ssl_protocols TLSv1.3;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Fowarded-For $proxy_add_x_forwarded_for;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
try_files $uri$args $uri$args/ /index.html;
}
location /twopyter/ {
# rewrite ^/hub/(.*) $scheme://$server_name/$1/$2 permanent;
# rewrite ^/twopyter/ /stupid break;
# rewrite (.*) $1 break;
# rewrite ^/twopyter/(.*)$ /$1 break;
rewrite /twopyter/(.*) ws://host.docker.internal:15080/twopyter/$1 break;
rewrite /twopyter/(.*) http://host.docker.internal:15080/twopyter/$1 break;
proxy_pass http://host.docker.internal:15080;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# websocket headers
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
JupyterHub Configuration
- Only shows uncommented vars
## The public facing URL of the whole JupyterHub application.
#
# This is the address on which the proxy will bind.
# Sets protocol, ip, base_url
# Default: 'http://:8000'
c.JupyterHub.bind_url = 'http://:8000/twopyter/'
## The class to use for spawning single-user servers.
#
# Should be a subclass of :class:`jupyterhub.spawner.Spawner`.
#
# .. versionchanged:: 1.0
# spawners may be registered via entry points,
# e.g. `c.JupyterHub.spawner_class = 'localprocess'`
#
# Currently installed:
# - default: jupyterhub.spawner.LocalProcessSpawner
# - localprocess: jupyterhub.spawner.LocalProcessSpawner
# - simple: jupyterhub.spawner.SimpleLocalProcessSpawner
# Default: 'jupyterhub.spawner.LocalProcessSpawner'
c.JupyterHub.spawner_class = 'jupyterhub.spawner.LocalProcessSpawner'
## Path to the notebook directory for the single-user server.
#
# The user sees a file listing of this directory when the notebook interface is
# started. The current interface does not easily allow browsing beyond the
# subdirectories in this directory's tree.
#
# `~` will be expanded to the home directory of the user, and {username} will be
# replaced with the name of the user.
#
# Note that this does *not* prevent users from accessing files outside of this
# path! They can do so with many other means.
# Default: ''
c.Spawner.notebook_dir = '~/'