Since I need to rebuild the docker images based on Amazon linux, I want to do the local test to make sure the image works. However, when I tried to run out of box of k8s-singleuser-sample docker image, and I also config the host port as 8888, even I can run it using docker command, however, I just can’t access the web console (I got the URL from the log – the last line). The error message is (I also tried localhost but got same error):
This page isn’t working
127.0.0.1 didn’t send any data.
ERR_EMPTY_RESPONSE
Could anyone let me know if I can run the singleuser-sample alone and locally? If I can, how to do that?
By the way, since I will also need to rebuild proxy and hub, could anyone let me know how to do the local test for those two images? I mean by just running the docker image locally, without deploying to EKS.
I’m pretty sure this is due to the server binding to localhost only so expose doesn’t work, on kubernetes networking is plumbed differently. To test locally with docker you should be able to override the command.
I just tested locally as follows
cd images/singleuser-sample
docker build -t k8s-singleuser-sample:local .
If you just run per the README.md you can check to see if it is actually running by docker exec -it CONTAINER_ID python and using requests to connect locally (the image doesn’t have curl or netstat I think).
Python 3.11.1 (main, Jan 23 2023, 21:18:23) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get("http://127.0.0.1:8888/")
<Response [200]>
To get it to bind correctly for local testing just give it the command setting the ip to 0.0.0.0
A quick question, could you let me know the image location? I am using Windows 10 with WSL 2, I searched everywhere but just can’t find the images. Also, if I run docker info, I will get
Docker Root Dir: /var/lib/docker
But apparently it’s not available on my windows machine.
In fact, if I understand correctly, I can just download the singleuser-sample dockerfile, and then run the build command suggested by you:
docker build -t k8s-singleuser-sample:local .
Will that work? If that works, could you confirm below is the correct dockerfile for singleuser-sample:
Also, how can tell if above link is version 2.0.0? If not, could you let me know how to get the dockerfile for the version 2.0.0?
To correctly build the image you will want to clone the repository so you have the requirements files the image build uses as well as the Dockerfile. If you absolutely want to build off the tag you can do:
git clone https://github.com/jupyterhub/zero-to-jupyterhub-k8s.git
cd zero-to-jupyterhub-k8s/images/singleuser-sample
git checkout 2.0.0
You shouldn’t have to worry about the local location of the image on disk - but to get this to work you’ll need to push the rebuilt image to a repository that your kubernetes cluster has access to and override the image in your helm config.yaml - see Customizing User Environment — Zero to JupyterHub with Kubernetes documentation
My instructions were written assuming you are using a UNIX shell to run, that takes a double dash -- to mean end of argument parsing.
I don’t have a Windows shell readily available to me to test. But this looks as if the shell you are using does not do this (you mentioned WSL so I assumed you might be using bash). Try either without the double dash
@sam123 given your goal is rebuilding images from a different base to run on EKS, that is not really using Zero to JupyterHub on Kubernetes.
I would probably recommend you build and test using the helm charts and a local kubernetes (minikube, k3s, kind).
I don’t think you are going to get the testing you want using just the docker images.
However you should be able to get each individual component running locally - for hub you probably want to provide a config.
Can you explain a bit more about the security issue you mention
If it is a specific CVE that needs a rebuild that wouldn’t need you to basically create your own images. I do see that there is an automatic scan for vulnerabilities that causes rebuilds - eg see this
Per your questions, it’s the company policy about docker base image (must base on Amazon Linux). It’s kind of strange requirement but leaves no choice for us except rebuild.
The local testing what I mean is to do the preliminary testing locally using docker (instead of deploy to EKS). For example, for singleuser, as long as I can run the docker image locally as a docker container and can access the Notebook web console, then I can do some functional testing to make sure the rebuilt docker image works (to some kind). Of course, eventually will deploy the rebuilt docker image to EKS for the finial testing.
Same thing for hub and proxy local testing. I just want to make sure I can run both locally using docker, and do some preliminary testing locally to make sure the docker image works and the main function works. After that, will deploy both to EKS for the final testing.
Please let me know for any further questions.
In that case, I would greatly appreciate if you could advice how to do the local test for hub and proxy–just need to run as docker container, then either ssh login the container or maybe there are some external tools that I can use to do some functional testing.
A quick question, after I start the singleuser-sample locally as a docker container, there is no authentication and I can go directly get to the web console, as a normal user. I know the hub will do the authentication part. My question is, if there is any simple settings to change on singleuser-sample to make the user as admin. I just want to test more locally.
That project has a docker-compose file which you can edit to change to the images you build (k8s-hub) and will allow you to test hub, proxy, and docker spawner.
You could test each component separately
To just smoke test the separate proxy container you can do the following:
That will test you’ve got an image with the configurable-http-proxy, the above tests the released version that z2jh uses, when you build replace jupyterhub/configurable-http-proxy:4.5.3 with your image name and tag. To build you should make a clone of the
To test the hub you should create a jupyterhub_config.py and mount that into the k8s-hub based image (eg /srv/jupyterhub/jupyterhub_config.py) you build and use docker to run jupyterhub --config
"""sample jupyterhub config file for testing
configures jupyterhub with dummyauthenticator and simplespawner
to enable testing without administrative privileges.
"""
c = get_config() # noqa
from jupyterhub.auth import DummyAuthenticator
c.JupyterHub.authenticator_class = DummyAuthenticator
# Optionally set a global password that all users must use
# c.DummyAuthenticator.password = "your_password"
from jupyterhub.spawner import SimpleLocalProcessSpawner
c.JupyterHub.spawner_class = SimpleLocalProcessSpawner
# only listen on localhost for testing
c.JupyterHub.bind_url = 'http://0.0.0.0:8083'
Another question about setup local environment for the testing and really appreciate your advise. If I setup a local environment, for example, minikube, for the local testing purpose. My concerns are:
Is there any instruction that I can follow to deploy JupyterHub to minikube? I don’t know if I still can follow the instructions from zero to jupyterhub on kubernetes, since those instructions are for K8S on cloud.
How much difference between minikube with EKS (K8S on AWS)? My goal is to deploy JupyterHub to EKS, instead of any local K8S. If there are lots of difference, may not worth to setup the local environment for the testing, because even everything works fine locally, may not work on EKS.
The main limitations for running locally with minikube are going to be the amount of resources you have to run (eg the RAM and CPU of your workstation or laptop). However you can use z2jh install instructions to install vial helm and it works just fine (I’ve been doing this with colima on OS X). As you mentioned running Windows, if you have sufficient you could look at microk8s Install MicroK8s on Windows | Ubuntu and follow the baremetal instructions to get running Kubernetes on a Bare Metal Host with MicroK8s — Zero to JupyterHub with Kubernetes documentation
The other way would be to set up a test k8s cluster using EKS that you can test and then destroy.
These are really decisions that you and your organization need to make around testing if you are rebuilding z2jh to run ontop of Amazon Linux rather than using the helm chart and containers built by the project.
The other alternative would be to ignore z2jh helm charts and manage your own kubernetes config, for that you might want to get a deeper understanding of setting up jupyterhub by hand such as (note I’ve not personally followed this guide, just pointing to options) GitHub - jupyterhub/jupyterhub-the-hard-way: Understanding JupyterHub by deploying it step by step - if you do this that’s outside of the scope of this forum topic for z2jh.
Note, you should take my replies here in context as suggestions from someone who has only recently started experimenting with z2jh.
I believe have answered your original question and as per the FAQ we should try keep this discussion tidy, and if you have new questions they should be separate posts.