Repo2Docker Image Caching

Hey Community,

I’m currently using repo2docker for building images based on my local source directory during development.

I have noticed that it copies the whole repo into the image and then runs pipenv install which can take several minutes.

Step 44/54 : COPY src/ ${REPO_DIR}

As most changes don’t involve dependency changes (in Pipfile or Pipfile.lock), I am curious if anybody has floated the idea of copying over these files first and installing before copying over the rest of the source so we can benefit from layer caching?

I may raise a PR for this myself at some point if there is no good reason why this has not already been done.

Many thanks

2 Likes

We have a mechanism for this in repo2docker with

It has been implemented for the conda, pip and R buidlpacks. Adding it to the Pipenv buildpack would be a nice contribution.

The trickiest part is detecting if a file which is used to install dependencies (possible) depends on the contents of the repository. For a requirements.txt we can scan for things like -r or -e . in the file. For install.R we implemented a mechanism to attempt to install things and back out if they failed. I am not sure what the right mechanism is for a Pipfile so your thoughts would be great.

Please do open a PR, even if it is just a sketch of an idea. It means others can keep building on it later :smiley:

1 Like