Persisting installed packages in JupyterLab running via Docker Compose

JupyterLab
1

Hi there,

I’m running one of the Jupyter Docker stacks, image name quay.io/jupyter/pytorch-notebook:cuda12-python-3.11.8. This is great, however I am finding one issue less convenient; being that the (pip) packages that I install are not persisted, so they need to be reinstalled every time I start up my containers again.

Solution sought:
A way to have pip package installed and persisted(!) when starting up the docker container for the first time.

What I’ve done so far:
I am running this docker container via a custom docker-compose.yml with this content:

services:
  jupyterlab:
    image: quay.io/jupyter/pytorch-notebook:cuda12-python-3.11.8
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    ports:
      - "8888:8888"
    working_dir: /home/jovyan
    volumes:
      - ./startup_hooks:/usr/local/bin/start-notebook.d/
      - ./src:/home/jovyan/src
    env_file:
      - .env
    command:
      - start-notebook.py
      - --PasswordIdentityProvider.hashed_password=${JP_PASS}

The volume mounted initiates a shell script that looks like:

#!/bin/bash
pip install -r /absolute_path_to/requirements.txt

This works fine but as stated every time I reboot the container, the pip packages have to be reinstalled all over again and this takes some time.

I’m assuming one possible way of solving this problem is to use a custom Dockerfile in which I simply RUN pip install ... however I can’t seem to find the Dockerfile being used in the image I currently use, nor information on how to use a custom docker file.

Any suggestions are greatly appreciated, thanks in advance!

2

While posting my question I come to realize some thing and after some more searching around I found a solution. Might be useful for others in the future so I’ll just leave this here;

The Dockerfiles used by the Jupyter Docker Stacks images can be found in their repository (of course, self shame);

I copied one over (docker-stacks/images/pytorch-notebook/cuda12/Dockerfile at 07dc2d08734b384ab7370956d47909bda9bc928b · jupyter/docker-stacks · GitHub), added this:

COPY requirements.txt /home/requirements.txt
RUN pip install -r /home/requirements.txt

Changed my docker-compose as follows:

services:
  jupyterlab:
    build: .
...

And it now uses the local Dockerfile! Thanks for being my rubber duck :wink:

1 Like