JupyterHub with Podman and DockerSpawner: Singleserver Containers can't communicate with outside

JupyterHub
, , ,
1

Hi guys,

I spwan singleuser containers using Jupyterhub with podman. The Singleservers are accessible, but can’t connect to the outside:

image
image1486×308 30.5 KB

Here the configuration file:

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect(("8.8.8.8", 80))

c.JupyterHub.hub_ip = s.getsockname()[0]
s.close()


c.JupyterHub.authenticator_class = "dummy"
c.Authenticator.admin_users = {'demo'}

c.JupyterHub.log_level = 'DEBUG'
c.ConfigurableHTTPProxy.debug = True

c.Authenticator.admin_users = {'Admin'}

c.JupyterHub.spawner_class = 'docker'
c.DockerSpawner.remove = True
c.DockerSpawner.default_url = '/lab'
c.DockerSpawner.start_timeout = 300
c.DockerSpawner.http_timeout = 120
c.DockerSpawner.use_internal_ip = True

c.DockerSpawner.network_name = 'jupyter_default'

c.DockerSpawner.debug = True

notebook_dir = os.environ.get('DOCKER_NOTEBOOK_DIR') or '/home/jovyan/work'
c.DockerSpawner.notebook_dir = notebook_dir

c.DockerSpawner.image = "jupyter/base-notebook:latest" #jupyter/datascience-notebook:latest docker pull jupyter/base-notebook

c.Spawner.cpu_limit = 1
c.Spawner.mem_limit = '8G'

This only occurs with containers ran with Jupyterhub. When I run the same image directly with podman:

services:
  jupyter:
    image: jupyter/base-notebook:latest
    container_name: JupyterLab_5
    restart: "always"
    ports:

      - "8886:8888"
    volumes:
      - /data/data_analysis:/home/jovyan/work/data_analysis

I run this compose file using podman compose. Here si how podman run the image:

podman create --name=JupyterLab_5 --label io.podman.compose.config-hash=123 
--label io.podman.compose.project=jupyter --label io.podman.compose.version=0.0.1 
--label com.docker.compose.project=jupyter --label com.docker.compose.project.working_dir=/home/rootx/jupyter 
--label com.docker.compose.project.config_files=docker-compose.yml --label com.docker.compose.container-number=1 
--label com.docker.compose.service=jupyter -v /data/data_analysis:/home/jovyan/work/data_analysis --net jupyter_default 
--network-alias jupyter -p 8886:8888 --restart always docker.io/jupyter/datascience-notebook

I could not find out why the containers run by Jupyterhub can’t access the outside. Hope someone can help. Thanks!

2

After you spawn a single user server from JupyterHub, could you try to spawn a shell into singleuser container and check for connectivity using curl. Something like

$ podman exec -it <singleuser_container> bash
$ curl -vvvv -I https://8.8.8.8
3

Hello Mahendra,

Thanks a lot! I spotted the difference with your command. For the singleserver spawned with Jupyterhub proxy variable (https_proxy, no_proxy) from the host are not being passed to the server (when running the singleserver directly with podman, they get passed)! And because we work behind a corporate proxy the Server can’t reach outside. When I logged in inside the container and added the proxy variables manually, it worked! Thanks a lot

2 Likes