Is this dirty build hack acceptable to use extensions on raspberry?

I am trying to use jupyterlab hub on raspberries. I have seen in the logs that the “build memory error” derived from a problem with webpack. It is impossible to build even when there is no extension. I have succeeded with a dirty trick, is it acceptable ? would it be possible to build jupyterlab that way properly ?

I have installed command line webpack webpack-cli webpack-bundle-analyzer, change directory to /usr/local/share/jupyter/lab/staging then execute :

webpack --config

which worked properly (or almost) yielding some informative messages on the assets size and enabling jupyterlab without error message and with available extensions :

     WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
    This can impact web performance.
      664de3932dd6291b4b8a8c0ddbcb4c61.svg (896 KiB)
      2f12242375edd68e9013ecfb59c672e9.svg (730 KiB)
      vendors~main.825fba72abae7de1679c.js (4.69 MiB)
      2.9ccb7977a199a4a50689.js (454 KiB)
      3.50663db94d152dab6429.js (883 KiB)
      4.69406a7d6939b46f4e98.js (253 KiB)
    WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
      main (4.71 MiB)

You can run jupyter lab build without any extensions, and it will just do essentially what you did manually with webpack. That’s a fully supported feature.

The standard command jupyter lab build aways fails even with pi4-4Gb without dev and without minimize, with the following error message :

 [LabBuildApp] JupyterLab 2.2.9
[LabBuildApp] Building in /usr/local/share/jupyter/lab
[LabBuildApp] Building jupyterlab assets (build:prod:minimize)
Build failed.

With explanation in the log :

Error: Command failed: /usr/local/share/jupyter/lab/staging/node_modules/.bin/webpack --config
    at checkExecSyncError (node:child_process:636:11)
    at Object.execFileSync (node:child_process:654:15)
    at Object.<anonymous> (/usr/local/share/jupyter/lab/staging/node_modules/@jupyterlab/buildutils/lib/ensure-max-old-space.js:38:17)
    at Module._compile (node:internal/modules/cjs/loader:1083:30)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1112:10)
    at Module.load (node:internal/modules/cjs/loader:948:32)
    at Function.Module._load (node:internal/modules/cjs/loader:789:14)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:72:12)
    at node:internal/main/run_main_module:17:47 {
  status: null,
  signal: 'SIGILL',
  output: [ null, null, null ],
  pid: 1572,
  stdout: null,
  stderr: null
error Command failed with exit code 1.
info Visit for documentation about this command.

[LabBuildApp] JupyterLab failed to build
[LabBuildApp] Traceback (most recent call last):
[LabBuildApp]   File "/usr/local/lib/python3.7/dist-packages/jupyterlab/", line 47, in debug_logging
[LabBuildApp]   File "/usr/local/lib/python3.7/dist-packages/jupyterlab/", line 138, in start
    raise e
[LabBuildApp]   File "/usr/local/lib/python3.7/dist-packages/jupyterlab/", line 135, in start
    command=command, app_options=app_options)
[LabBuildApp]   File "/usr/local/lib/python3.7/dist-packages/jupyterlab/", line 460, in build
    command=command, clean_staging=clean_staging)
[LabBuildApp]   File "/usr/local/lib/python3.7/dist-packages/jupyterlab/", line 661, in build
    raise RuntimeError(msg)
[LabBuildApp] RuntimeError: JupyterLab failed to build
[LabBuildApp] Exiting application: JupyterLab

Am I doing something wrong ?


Does jupyter lab build --minimize=False work? It may be that you run out of memory.

It is a little odd that it seems that running webpack manually works, but running with jupyter lab build doesn’t. It may have to do with how we try to automatically do memory management for old node versions (will be gone in jlab 4, and maybe in jlab 3).

What if you do this to set the space node uses to 3GB (it defaults to 4GB, perhaps that is where it is running into issues)

NODE_OPTIONS=--max-old-space-size=3000 jupyter lab build

I just submitted, which hopefully solves this problem for jlab 3.

1 Like

No it fails with the same error even if there is no extension.

It does not work either but I got the memory error with the “direct” method when I added multiple extensions (not so many, unfortunately… mainly because of some core assets being huge compared to the recommended limit ) :

webpack --config
<--- Last few GCs --->
[2246:0x4640bc0]   170951 ms: Mark-sweep 756.0 (867.1) -> 744.1 (839.6) MB, 600.6 / 0.0 ms  (average mu = 0.898, current mu = 0.771) allocation failure scavenge might not succeed
<--- JS stacktrace --->
FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory

And I was able to overcome it with following command :

NODE_OPTIONS="--max-old-space-size=3072" webpack --config

which means that you are right on both your affirmations, I noticed that the error at Object.<anonymous> appear when webpack is not started from /usr/local/share/jupyter/lab/staging and do not find the file index.js in case it helps to find a solution.

Many thanks @jasongrout I’ll test it ASAP and send feedback, at the moment I have to work on the raspi Os releases with jupyterlab not sending warning messages included.

Sorry @jasongrout to come back so late. Here are some news after jlab3 upgrade or full install :

  • the installation or upgrade works perfectly :+1:
  • lab build works perfectly with raspberry Pi4 outside lab
balfourdowns-€ : sudo systemctl stop jupyterlabhub
vidal[~] balfourdowns-€ : sudo jupyter lab build
[LabBuildApp] JupyterLab 3.0.5
[LabBuildApp] Building in /usr/local/share/jupyter/lab
[LabBuildApp] Building jupyterlab assets (production, minimized)
  • from inside lab sometimes build fails but that may be caused by bad sources in the extension menu (some cleaning in there might be very usefull :zipper_mouth_face: :innocent:)
  • Sstandard build still fails with raspberry3 :sob:with the memory heap bug :scream: . I know that JS is fantastic but it is also evil; there is a new version every second day, to compile it steals all your memory and your neighbour’s also, nobody cares about backwards compatibility… But a fix is necessary for our low cost, low energy, low memory tools.
  • I have released binary debian images with pre-compiled preinstalled jupyterhub lab service started at boot. If you are the lucky owner of a raspi or know one who has got one; you may test from one of the images provided here. I am preparing a page in english before publicising it.

Here is the output from lab build with webpack and the above trick

NODE_OPTIONS="--max-old-space-size=850" webpack --config
(node:3524) [DEP_WEBPACK_MAIN_TEMPLATE_GET_ASSET_PATH] DeprecationWarning: MainTemplate.getAssetPath is deprecated (use Compilation.getAssetPath instead)
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:3524) [DEP_WEBPACK_MAIN_TEMPLATE_GET_PUBLIC_PATH] DeprecationWarning: MainTemplate.getPublicPath is deprecated (use Compilation.getAssetPath(compilation.outputOptions.publicPath, options) instead)
(node:3524) [DEP_WEBPACK_COMPILATION_ASSETS] DeprecationWarning: Compilation.assets will be frozen in future, all modifications are deprecated.
BREAKING CHANGE: No more changes should happen to Compilation.assets after sealing the Compilation.
	Do changes to assets earlier, e. g. in Compilation.hooks.processAssets.
	Make sure to select an appropriate stage from Compilation.PROCESS_ASSETS_STAGE_*.
(node:3524) [DEP_WEBPACK_MODULE_ISSUER] DeprecationWarning: Module.issuer: Use new ModuleGraph API
assets by path *.js 7.23 MiB
  assets by chunk 4.56 MiB (id hint: vendors)
    assets by status 3.47 MiB [big] 8 assets
    18 assets
assets by chunk 2.81 MiB (auxiliary id hint: vendors)
  assets by path *.svg 1.73 MiB 3 assets
  assets by path *.eot 367 KiB 3 assets
  assets by path *.ttf 366 KiB 3 assets
  assets by path *.woff 208 KiB 3 assets
  assets by path *.woff2 168 KiB
    asset 3ceb50e7bcafb577367c21b53064e00c.woff2 78.4 KiB [emitted] (auxiliary id hint: vendors)
    asset c967a94cfbe2b06627ffe7861c741a61.woff2 76.6 KiB [emitted] (auxiliary id hint: vendors)
    asset 1008b5226941c24f44681061d347ad5e.woff2 13.2 KiB [emitted] (auxiliary id hint: vendors)
asset index.html 1.53 KiB [emitted]
orphan modules 4.67 MiB [orphan] 1233 modules
runtime modules 97.8 KiB 14 modules
built modules 13.3 MiB (javascript) 4.43 KiB (share-init) 4.43 KiB (consume-shared) [built]
  modules by path ./node_modules/ 13.3 MiB
    cacheable modules 13.3 MiB 977 modules
    3 modules
  provide-module modules 4.43 KiB 108 modules
  consume-shared-module modules 4.43 KiB
    modules by path consume shared module (default) @jupyterlab/ 3.53 KiB 41 modules
    modules by path consume shared module (default) @lumino/ 588 bytes 14 modules
  modules by path ./build/ 22.7 KiB
    ./build/bootstrap.js 3.78 KiB [built] [code generated]
    ./build/index.out.js + 1 modules 18.9 KiB [built] [code generated]
  4 modules

WARNING in ./node_modules/vega-lite/build/src/index.js 5:0-30
Should not import the named export 'version' (reexported as 'version') from default-exporting module (only default export is available soon)
 @ consume shared module (default) vega-lite@^4.0.2 (strict) (fallback: ./node_modules/vega-lite/build/src/index.js)
 @ ./node_modules/vega-embed/build/vega-embed.module.js 6:0-44 189:15-29
 @ consume shared module (default) vega-embed@^6.2.1 (strict) (fallback: ./node_modules/vega-embed/build/vega-embed.module.js)
 @ ./node_modules/@jupyterlab/vega5-extension/lib/index.js 166:28-48
 @ consume shared module (default) @jupyterlab/vega5-extension@~3.0.2 (strict) (fallback: ./node_modules/@jupyterlab/vega5-extension/lib/index.js)
 @ ./build/index.out.js 153:16-54
 @ ./build/bootstrap.js

WARNING in vega-expression
  Multiple versions of vega-expression found:
    3.0.1 ./~/vega-lite/~/vega-expression from ./~/vega-lite/build/src/compile/data/expressions.js
    4.0.1 ./~/vega-expression from ./~/vega-functions/build/vega-functions.module.js

Check how you can resolve duplicate packages:

WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
  e33e2cf6e02cac2ccb77689015f11c56.svg (730 KiB)
  371dbce0dd46bd4d2033eb2334f28580.svg (896 KiB)
  9741.b5e38b048132c3b57b8b.js (511 KiB)
  25.282b82e1d552316d115a.js (303 KiB)
  jlab_core.690a398af0cc8e4bdaeb.js (1.96 MiB)
  714.286e0fa39e35ace9082c.js (292 KiB)
  1281.d6b5b4c4adb657ccc9a7.js (450 KiB)
  807.b65d521f16651a48f0eb.js (744 KiB)
  9473.ded9e7826e2dfdaf9757.js (315 KiB)
  2719.28123e57f2abeb3891b7.js (687 KiB)
  2156.da63a4f0e2cc20860144.js (253 KiB)

webpack 5.13.0 compiled with 3 warnings in 522775 ms

asset index.css 16.1 KiB [compared for emit] (name: index)
asset index.js 0 bytes [compared for emit] [minimized] (name: index)
Entrypoint index 16.1 KiB = index.css 16.1 KiB index.js 0 bytes
./node_modules/@jupyterlab/theme-dark-extension/style/index.css 50 bytes [built] [code generated]
css ./node_modules/css-loader/dist/cjs.js!./node_modules/@jupyterlab/theme-dark-extension/style/variables.css 15.3 KiB [code generated]
css ./node_modules/css-loader/dist/cjs.js!./node_modules/@jupyterlab/theme-dark-extension/style/base.css 472 bytes [code generated]
css ./node_modules/css-loader/dist/cjs.js!./node_modules/@jupyterlab/theme-dark-extension/style/index.css 342 bytes [code generated]
webpack 5.13.0 compiled successfully in 25038 ms

asset index.css 15 KiB [compared for emit] (name: index)
asset index.js 0 bytes [compared for emit] [minimized] (name: index)
Entrypoint index 15 KiB = index.css 15 KiB index.js 0 bytes
./node_modules/@jupyterlab/theme-light-extension/style/index.css 50 bytes [built] [code generated]
css ./node_modules/css-loader/dist/cjs.js!./node_modules/@jupyterlab/theme-light-extension/style/variables.css 14.2 KiB [code generated]
css ./node_modules/css-loader/dist/cjs.js!./node_modules/@jupyterlab/theme-light-extension/style/base.css 472 bytes [code generated]
css ./node_modules/css-loader/dist/cjs.js!./node_modules/@jupyterlab/theme-light-extension/style/index.css 342 bytes [code generated]
webpack 5.13.0 compiled successfully in 24982 ms

Thanks for your help and congrats for jlab3 it is really top !

Thanks for following up. I’m glad jlab 3 is working great!