I recently updated JupyterLab to 1.2.1. As I was adding some extensions, I noticed that the building of them was really slow, like 20 minutes on my T495 Ubuntu machine.
I installed JupyterLab through conda
# Name Version Build Channel
_libgcc_mutex 0.1 main
appdirs 1.4.3 py_1 conda-forge
attrs 19.3.0 py_0 conda-forge
backcall 0.1.0 py_0 conda-forge
black 19.10b0 py37_0 conda-forge
bleach 3.1.0 py_0 conda-forge
bzip2 1.0.8 h516909a_1 conda-forge
ca-certificates 2019.9.11 hecc5488_0 conda-forge
certifi 2019.9.11 py37_0 conda-forge
click 7.0 py_0 conda-forge
colorama 0.4.1 py_0 conda-forge
decorator 4.4.1 py_0 conda-forge
defusedxml 0.6.0 py_0 conda-forge
entrypoints 0.3 py37_1000 conda-forge
icu 64.2 he1b5a44_1 conda-forge
importlib_metadata 0.23 py37_0 conda-forge
ipykernel 5.1.3 py37h5ca1d4c_0 conda-forge
ipython 7.9.0 py37h5ca1d4c_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
jedi 0.15.1 py37_0 conda-forge
jinja2 2.10.3 py_0 conda-forge
json5 0.8.5 py_0 conda-forge
jsonschema 3.1.1 py37_0 conda-forge
jupyter_client 5.3.3 py37_1 conda-forge
jupyter_core 4.5.0 py_0 conda-forge
jupyterlab 1.2.1 py_0 conda-forge
jupyterlab-black 0.2.1 pypi_0 pypi
jupyterlab_code_formatter 0.7.0 py_0 conda-forge
jupyterlab_server 1.0.6 py_0 conda-forge
libffi 3.2.1 he1b5a44_1006 conda-forge
libgcc-ng 9.1.0 hdf63c60_0
libsodium 1.0.17 h516909a_0 conda-forge
libstdcxx-ng 9.1.0 hdf63c60_0
libuv 1.33.1 h516909a_0 conda-forge
markupsafe 1.1.1 py37h516909a_0 conda-forge
mistune 0.8.4 py37h516909a_1000 conda-forge
more-itertools 7.2.0 py_0 conda-forge
mypy_extensions 0.4.3 py37_0 conda-forge
nbcommands 0.3.2 py_0 conda-forge
nbconvert 5.6.1 py37_0 conda-forge
nbformat 4.4.0 py_1 conda-forge
ncurses 6.1 hf484d3e_1002 conda-forge
nodejs 12.13.0 h10a4023_0 conda-forge
notebook 6.0.1 py37_0 conda-forge
openssl 1.1.1c h516909a_0 conda-forge
packaging 19.2 py_0 conda-forge
pandoc 2.7.3 0 conda-forge
pandocfilters 1.4.2 py_1 conda-forge
parso 0.5.1 py_0 conda-forge
pathspec 0.6.0 py_0 conda-forge
pexpect 4.7.0 py37_0 conda-forge
pickleshare 0.7.5 py37_1000 conda-forge
pip 19.3.1 py37_0 conda-forge
prometheus_client 0.7.1 py_0 conda-forge
prompt_toolkit 2.0.10 py_0 conda-forge
ptyprocess 0.6.0 py_1001 conda-forge
pygments 2.4.2 py_0 conda-forge
pyparsing 2.4.4 py_0 conda-forge
pyrsistent 0.15.5 py37h516909a_0 conda-forge
python 3.7.3 h33d41f4_1 conda-forge
python-dateutil 2.8.1 py_0 conda-forge
pyzmq 18.1.0 py37h1768529_0 conda-forge
readline 8.0 hf8c457e_0 conda-forge
regex 2019.11.1 py37h516909a_0 conda-forge
send2trash 1.5.0 py_0 conda-forge
setuptools 41.6.0 py37_1 conda-forge
six 1.13.0 py37_0 conda-forge
sqlite 3.30.1 hcee41ef_0 conda-forge
terminado 0.8.2 py37_0 conda-forge
testpath 0.4.4 py_0 conda-forge
tk 8.6.9 hed695b0_1003 conda-forge
toml 0.10.0 py_0 conda-forge
tornado 6.0.3 py37h516909a_0 conda-forge
traitlets 4.3.3 py37_0 conda-forge
typed-ast 1.4.0 py37h516909a_0 conda-forge
typing_extensions 126.96.36.199 py37_0 conda-forge
wcwidth 0.1.7 py_1 conda-forge
webencodings 0.5.1 py_1 conda-forge
wheel 0.33.6 py37_0 conda-forge
xz 5.2.4 h14c3975_1001 conda-forge
zeromq 4.3.2 he1b5a44_2 conda-forge
zipp 0.6.0 py_0 conda-forge
zlib 1.2.11 h516909a_1006 conda-forge
and I am installing the extensions through the following bash script
# Choose conda and load the virtual environment
CONDA_BASE=$(conda info --base)
conda activate jupyter-lab
# Node JS for the extensions
conda install -c conda-forge nodejs
conda install -c conda-forge nbcommands
# Jupyter Lab code formatter with black
conda install -c conda-forge black jupyterlab_code_formatter
jupyter labextension install @ryantam626/jupyterlab_code_formatter
jupyter serverextension enable --py jupyterlab_code_formatter
# Jupyter widgets extension, FigureWidget and renderer support
jupyter labextension install @email@example.com --no-build
jupyter labextension install firstname.lastname@example.org --no-build
jupyter labextension install email@example.com --no-build
# Go To Definition extension
jupyter labextension install @krassowski/jupyterlab_go_to_definition --no-build
# Spell checker extension
jupyter labextension install @ijmbarr/jupyterlab_spellchecker --no-build
# Build extensions
jupyter lab build
I don’t remember anything particularly slow prior to the update. Right now, the installation lags at
jupyter labextension install @ryantam626/jupyterlab_code_formatter or
jupyter lab build.
Is the script flawed in some way? Would it be related to this post?
Thanks in advance for your help!
I’m just guessing, could it be a network issue? If you run jupyter lab build --debug is there a timeout happening somewhere?
I’m seeing the same problem. Extremely slow build time for the jupyterlab extensions.
I’m running JupyterLab v1.2.3
I only have three extensions to add, and am running the following (as per the plotly documentation):
# Jupyter widgets extension
jupyter labextension install @firstname.lastname@example.org --no-build
# jupyterlab renderer support
jupyter labextension install email@example.com --no-build
# FigureWidget support
jupyter labextension install firstname.lastname@example.org --no-build
# Build extensions (must be done to activate extensions since --no-build is used above)
jupyter lab build
Not only is it slow, but it then crashes with an error 137
The hanging seems to be taking place when installing jupyterlab-plotly.
> /usr/bin/npm pack jupyterlab-plotly
npm notice 📦 email@example.com
npm notice === Tarball Contents ===
npm notice 1.0kB package.json
npm notice 165B README.md
npm notice 545B style/index.css
npm notice 1.0kB style/plotly.svg
npm notice === Tarball Details ===
npm notice name: jupyterlab-plotly
npm notice version: 1.3.0
npm notice filename: jupyterlab-plotly-1.3.0.tgz
npm notice package size: 3.1 kB
npm notice unpacked size: 9.0 kB
npm notice shasum: bb648b175b99b6a3ac4784e00372e1056b625f1c
npm notice integrity: sha512-7goT3ELiFe1yO[...]Tr5fr1fGlOxDg==
npm notice total files: 5
Yarn configuration loaded.
Building jupyterlab assets (build:prod:minimize)
> node /usr/local/lib/python3.7/site-packages/jupyterlab/staging/yarn.js install --non-interactive
-yarn install v1.15.2
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info firstname.lastname@example.org: The platform "linux" is incompatible with this module.
info "email@example.com" is an optional dependency and failed compatibility check. Excluding it from installation.
[4/5] Linking dependencies...
[5/5] Building fresh packages...
success Saved lockfile.
Done in 7.19s.
> node /usr/local/lib/python3.7/site-packages/jupyterlab/staging/yarn.js yarn-deduplicate -s fewer
/yarn run v1.15.2
$ /usr/local/share/jupyter/lab/staging/node_modules/.bin/yarn-deduplicate -s fewer
Done in 0.63s.
> node /usr/local/lib/python3.7/site-packages/jupyterlab/staging/yarn.js run build:prod:minimize
And it hangs there forever before finally timing out
I am getting the distinct feeling this is related to plotly. There is some on-going work to resolve some of the likely root cause issues, namely the large size, and potential duplication, of the plotly distribution which gets injected into the main bundle along with lab itself.
This is partially on the Lab side, for making it possible for extensions to break the build so hard, and we’ve been doing what we can to try to alleviate some of the issues that core creates itself. It’s partially up to extension authors to be mindful of the total impact of their dependencies on all of their end users.
In the near term, an effective pattern is, where possible, to use
await import for very large packages, even if it means making wrappers and whatnot, and trying to delay the loading of these packages until they are actually asked for by the client, e.g. actually plotting a plot. Aside from helping Lab start faster, it means that the high-water mark of memory used during build is reduced.
There are some additional techniques we could employ, which we have been thus far avoiding, as they would multiply the number of individual files downloaded ten-fold, and HTTP 1.1 is not so great at that kind of request work load. So we’ll have to try to find a happy medium.
In the further future, we’d very much like to get to where we can handle the simultaneous challenges of proper deduplication of assets served to the browser (which is why we went with current webpack approach), and the efficient distribution, installation and serving of extensions without
webpack on end users machines (favoring
conda, or system package managers), while still allowing developers to use the millions of packages on
npm… but that’s a pretty advanced research topic at this point.
Given that the root cause may well be
plotlywidget, one thing you can try is running the build without minification:
$ jupyter lab build --minimize=False
That will explode your build size, so if you’re running on a remote server then this will cause browser load times to increase. However, if you’re running JupyterLab locally (either same-machine or same-network), then you shouldn’t feel very much impact at all.
Thank you all for your help and insights. It seems to me that the lagging was not only from
plot.ly. I tried the
--minimize=false option and, apparently, it reduces the setup time.
Perhaps unrelated, but the CLI parsing here is case-sensitive which means --minimize=False works but --minimize=false does not. At least it fails with a descriptive error, but I would have expected that either should work.