The way I find possible is to build a wheel package locally, then create a Dockerfile to build the image with jupyter/base-notebook as the base image and install the built wheel package in the Dockerfile. Other people with more expertise on this please correct me and add more information.
However, after modifying and building source under packages directory, we have to manually copy the built file into staging directory for it to be packaged into wheel package.
For example, after modifying files under packages/docregistry, we should do:
jlpm run build
cp -r packages/docregistry/lib jupyterlab/staging/node_modules/@jupyterlab/docregistry/
python -m build
There were discussions about this in the following threads also: