How can I build jupyterlab faster and reduce vendors~main.js?

JupyterLab
1

I tried expanding RAM space and it finally built successfully. So perhaps it is because of RAM space.

Every time I run jupyter lab build it takes hours to build and when it is built, it takes 3-5min for chrome to load jupyter lab pages.
I analyzed the pages and found out a vendors~main*******.js file is as big as 32.3 MB which I think is created by webpack.
I went to github issues and tried as what is mentioned. The problem is still there.
So I wonder how to make jupyterlab build faster and create vendors~main.js with smaller size.
Thanks for your help if any:).

2

It should not be taking literal hours, can you tell us more about your setup? Are you running dev builds (the default) or prod builds?

In our setup on a EC2 t2.small server, JupyterLab is able to build with 4 extensions (one of them quite meaty) in ~1.5min.

One thing that might be impacting you is #6804, which is fixed on master but not yet released. Try the workaround I describe there and see if it helps.

1 Like
3

Thanks for your reply. Here’s how I install jupyterlab and hope it helps to locate the problem.

  1. pip install jupyterlab and pip install jupyter
  2. wget {url for latest npm} and setup ln -s for npm and node, the version ofr node is 12.x
  3. jupyter lab install(failed)
  4. I looked into error log. It said I don’t have python2 since I am running on Ubuntu 18.04. So I run apt install python
  5. jupyter lab install took more than 2hours to finish so I rebooted my computer and changed webpack --config webpack.prod.config.json to webpack in package.json. It still took long time to build but finally succeeded.
  6. jupyter lab and open chrome, the vendors~main.js shows up as big as 32MB.

I will try what you said in #6804 and will tell you whether it works.
Thanks again for your reply and help.

4

You could also try the fix at https://github.com/jupyterlab/jupyterlab/issues/3561#issuecomment-517673508 - use --dev-build=False when installing something

5

I tried jupyter lab build --dev-build=False and my computer is still running ( more than one hour) so it seems dev-build=False doesn’t work. Do you mean jupyter lab build --dev-build=False or jupyter labextension install XXXX --dev-build=False?

6

Can you try the new alpha on pypi, 1.1.0a1 ? It has some fixes merged to build smaller builds by default.

pip install --pre --upgrade jupyterlab
7

FYI, here’s log content after trying jupyter lab build --dev-build=False and source-map but failed.

  1 [LabBuildApp] Building in /usr/local/share/jupyter/lab
  2 [LabBuildApp] Node v12.7.0
  3
  4 [LabBuildApp] Building jupyterlab assets
  5 [LabBuildApp] > node /usr/local/lib/python3.6/dist-packages/jupyterlab/staging/yarn.js install --non-interactive
  6 [LabBuildApp] yarn install v1.15.2
  7 [1/5] Validating package.json...
  8 [2/5] Resolving packages...
  9 success Already up-to-date.
 10 Done in 0.89s.
 11
 12 [LabBuildApp] > node /usr/local/lib/python3.6/dist-packages/jupyterlab/staging/yarn.js yarn-deduplicate -s fewer --fail
 13 [LabBuildApp] yarn run v1.15.2
 14 $ /usr/local/share/jupyter/lab/staging/node_modules/.bin/yarn-deduplicate -s fewer --fail
 15 Done in 0.41s.
 16
 17 [LabBuildApp] > node /usr/local/lib/python3.6/dist-packages/jupyterlab/staging/yarn.js run build:prod
 18 [LabBuildApp] yarn run v1.15.2
 19 $ cross-env NODE_OPTIONS=--max_old_space_size=4096 webpack --config webpack.prod.config.js
 20 error Command failed with exit code 1.
 21 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
 22                                                                                                                                                           23 [LabBuildApp] JupyterLab failed to build
 24 [LabBuildApp] Traceback (most recent call last):                                                                                                          25
 26 [LabBuildApp]   File "/usr/local/lib/python3.6/dist-packages/jupyterlab/debuglog.py", line 47, in debug_logging                                           27     yield                                                                                                                                                 28
 29 [LabBuildApp]   File "/usr/local/lib/python3.6/dist-packages/jupyterlab/labapp.py", line 80, in start                                                     30     command=command, logger=self.log)
 31                                                                                                                                                           32 [LabBuildApp]   File "/usr/local/lib/python3.6/dist-packages/jupyterlab/commands.py", line 373, in build
 33     command=command, clean_staging=clean_staging)                                                                                                         34                                                                                                                                                           35 [LabBuildApp]   File "/usr/local/lib/python3.6/dist-packages/jupyterlab/commands.py", line 566, in build
 36     raise RuntimeError(msg)                                                                                                                               37
 38 [LabBuildApp] RuntimeError: JupyterLab failed to build
 39
 40 [LabBuildApp] Exiting application: JupyterLab
8

I tried to update jupyterlab it doesn’t work:joy:
I looked into log file. It should be a webpack problem. Perhaps the config file coursed the problem.
webpack.prod.config.js

  1 var merge = require('webpack-merge');
  2 var config = require('./webpack.config');
  3
  4 config[0] = merge(config[0], {
  5   mode: 'production',
  6   devtool: 'source-map',
  7   optimization: {
  8     minimize: false
  9   }
 10 });
 11
 12 module.exports = config;
9

I’m suffering from this issue with exactly the same symptoms and error message.
Running on a EC2 t2.micro, Ubuntu.
Fresh install of jupyterlab==1.1.4,
jupyter-client==5.3.3
jupyter-core==4.5.0
jupyterhub==1.0.0
jupyterlab==1.1.4
jupyterlab-server==1.0.6

All extensions uninstalled.
Install ipywidgets for lab with jupyter labextension install @jupyter-widgets/jupyterlab-manager@1.0 --no-build
Executed jupyter lab clean and
export npm_config_max_old_space_size=4096

sudo -E jupyter lab build runs interminably long, pegs the CPU and eventually hoses the EC2 instance requiring reboot.

10

For lack of any other alternative, now trying with an EC2 t2.small.

Is there a minimum RAM requirement?

11

That will probably depend a lot on how many and which extensions you install.

You might try building with --minimize=False, which may cut down on RAM usage.

1 Like
12

Thanks, I’ll give that a shot. Only a single extension (ipywidgets). Until I can get that working, I won’t bother with any others.

13

That’s the ticket! Running on a t2.small with --minimize=False (and all other previously recited incantations) build completes successfully in 51s and the extension operates as expected.
Thanks for the guidance.

14

Took me about about 10min to build Plotly requirements listed here

1 Like
15

Same, I came here worried it was stuck somewhere but it wasn’t lol