Okay, I solved it by doing the following in my Dockerfile:
# Install the extensions globally via conda
RUN conda install -c conda-forge jupyterlab ipywidgets plotly
# Build JupyterLab to activate the extensions
RUN jupyter lab build
These extensions were already installed in my environment, but building JupyterLab at the base layer solved the problem.