hi guys.
Trying to run a jupyter notebook on my Flink Jobmanager, The intend here is to write some Python based code on my Flink cluster via the ‘apache-flink’ module.
Below is my Dockerfile where I add the jupyter package and then the execution stack where i start it…
problem, when I go to the end point, it’s a big empty blank page.
Dockerfile
ARG REPO_NAME="georgelza"
FROM ${REPO_NAME}/apacheflink-base:1.20.1-scala_2.12-java17
SHELL ["/bin/bash", "-c"]
# this is where we have flink itself installed.
WORKDIR /opt/flink
ENV FLINK_HOME=/opt/flink
RUN mkdir -p /opt/flink/conf/
RUN mkdir -p /opt/flink/checkpoints
RUN mkdir -p /opt/flink/rocksdb
RUN mkdir -p /opt/sql-client/conf/
RUN chown -R flink:flink /opt/flink
RUN chown -R flink:flink /opt/sql-client
# Jupyter -> Temporary, for development purposes only, remember to add port 8888 to the docker compose export
# https://docs.jupyter.org/en/latest/running.html
RUN echo "--> Install apache-flink && jupyter package" \
&& /usr/bin/pip3 install jupyter
USER flink:flink
RUN echo "--> Install JARs: Flink's S3 plugin" && \
mkdir ./plugins/s3-fs-hadoop && \
mv ./opt/flink-s3-fs-hadoop-1.20.1.jar ./plugins/s3-fs-hadoop/
# Install JARs
# See https://repo.maven.apache.org/maven2/org/apache/flink/
RUN chown -R flink:flink /opt/flink
CMD ./bin/start-cluster.sh && sleep infinity
EXPOSE 8888
Docker Compose
jobmanager:
image: georgelza/apacheflink:1.20.1-scala_2.12-java17
hostname: jobmanager
container_name: jobmanager
ports:
- 8083:8081
- 8888:8888 # Jupyter notebook
environment:
- env.log.level=DEBUG
volumes:
- ./data/paimon:/tmp/paimon
- ./creFlinkFlows:/sql
- ./pyFlink:/pyapp
Starting server
make jm
docker compose exec jobmanager /bin/bash
flink@jobmanager:~$ jupyter notebook
[I 2025-05-17 09:56:20.988 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2025-05-17 09:56:20.990 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2025-05-17 09:56:20.993 ServerApp] jupyterlab | extension was successfully linked.
[I 2025-05-17 09:56:20.995 ServerApp] notebook | extension was successfully linked.
[I 2025-05-17 09:56:20.996 ServerApp] Writing Jupyter server cookie secret to /opt/flink/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2025-05-17 09:56:21.166 ServerApp] notebook_shim | extension was successfully linked.
[I 2025-05-17 09:56:21.181 ServerApp] notebook_shim | extension was successfully loaded.
[I 2025-05-17 09:56:21.183 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2025-05-17 09:56:21.184 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2025-05-17 09:56:21.186 LabApp] JupyterLab extension loaded from /usr/local/lib/python3.10/dist-packages/jupyterlab
[I 2025-05-17 09:56:21.186 LabApp] JupyterLab application directory is /usr/local/share/jupyter/lab
[I 2025-05-17 09:56:21.186 LabApp] Extension Manager is 'pypi'.
[I 2025-05-17 09:56:21.243 ServerApp] jupyterlab | extension was successfully loaded.
[I 2025-05-17 09:56:21.246 ServerApp] notebook | extension was successfully loaded.
[I 2025-05-17 09:56:21.247 ServerApp] Serving notebooks from local directory: /opt/flink
[I 2025-05-17 09:56:21.247 ServerApp] Jupyter Server 2.16.0 is running at:
[I 2025-05-17 09:56:21.247 ServerApp] http://localhost:8888/tree?token=df91905d3836954ab07a4b60691c6b6dd367c9bfba57d841
[I 2025-05-17 09:56:21.247 ServerApp] http://127.0.0.1:8888/tree?token=df91905d3836954ab07a4b60691c6b6dd367c9bfba57d841
[I 2025-05-17 09:56:21.247 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[W 2025-05-17 09:56:21.250 ServerApp] No web browser found: Error('could not locate runnable browser').
[C 2025-05-17 09:56:21.251 ServerApp]
To access the server, open this file in a browser:
file:///opt/flink/.local/share/jupyter/runtime/jpserver-1461-open.html
Or copy and paste one of these URLs:
http://localhost:8888/tree?token=df91905d3836954ab07a4b60691c6b6dd367c9bfba57d841
http://127.0.0.1:8888/tree?token=df91905d3836954ab07a4b60691c6b6dd367c9bfba57d841
[I 2025-05-17 09:56:21.262 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server