I am trying to use jupyterlab hub on raspberries. I have seen in the logs that the “build memory error” derived from a problem with webpack. It is impossible to build even when there is no extension. I have succeeded with a dirty trick, is it acceptable ? would it be possible to build jupyterlab that way properly ?
I have installed command line webpackwebpack-cliwebpack-bundle-analyzer, change directory to /usr/local/share/jupyter/lab/staging then execute :
webpack --config webpack.prod.minimize.config.js
which worked properly (or almost) yielding some informative messages on the assets size and enabling jupyterlab without error message and with available extensions :
WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets:
664de3932dd6291b4b8a8c0ddbcb4c61.svg (896 KiB)
2f12242375edd68e9013ecfb59c672e9.svg (730 KiB)
vendors~main.825fba72abae7de1679c.js (4.69 MiB)
2.9ccb7977a199a4a50689.js (454 KiB)
3.50663db94d152dab6429.js (883 KiB)
4.69406a7d6939b46f4e98.js (253 KiB)
WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
main (4.71 MiB)
vendors~main.825fba72abae7de1679c.js
main.4363983affea8feaa5a7.js
You can run jupyter lab build without any extensions, and it will just do essentially what you did manually with webpack. That’s a fully supported feature.
The standard command jupyter lab build aways fails even with pi4-4Gb without dev and without minimize, with the following error message :
[LabBuildApp] JupyterLab 2.2.9
[LabBuildApp] Building in /usr/local/share/jupyter/lab
[LabBuildApp] Building jupyterlab assets (build:prod:minimize)
Build failed.
.../...
With explanation in the log :
Error: Command failed: /usr/local/share/jupyter/lab/staging/node_modules/.bin/webpack --config webpack.prod.minimize.config.js
at checkExecSyncError (node:child_process:636:11)
at Object.execFileSync (node:child_process:654:15)
at Object.<anonymous> (/usr/local/share/jupyter/lab/staging/node_modules/@jupyterlab/buildutils/lib/ensure-max-old-space.js:38:17)
at Module._compile (node:internal/modules/cjs/loader:1083:30)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1112:10)
at Module.load (node:internal/modules/cjs/loader:948:32)
at Function.Module._load (node:internal/modules/cjs/loader:789:14)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:72:12)
at node:internal/main/run_main_module:17:47 {
status: null,
signal: 'SIGILL',
output: [ null, null, null ],
pid: 1572,
stdout: null,
stderr: null
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
[LabBuildApp] JupyterLab failed to build
[LabBuildApp] Traceback (most recent call last):
[LabBuildApp] File "/usr/local/lib/python3.7/dist-packages/jupyterlab/debuglog.py", line 47, in debug_logging
yield
[LabBuildApp] File "/usr/local/lib/python3.7/dist-packages/jupyterlab/labapp.py", line 138, in start
raise e
[LabBuildApp] File "/usr/local/lib/python3.7/dist-packages/jupyterlab/labapp.py", line 135, in start
command=command, app_options=app_options)
[LabBuildApp] File "/usr/local/lib/python3.7/dist-packages/jupyterlab/commands.py", line 460, in build
command=command, clean_staging=clean_staging)
[LabBuildApp] File "/usr/local/lib/python3.7/dist-packages/jupyterlab/commands.py", line 661, in build
raise RuntimeError(msg)
[LabBuildApp] RuntimeError: JupyterLab failed to build
[LabBuildApp] Exiting application: JupyterLab
It is a little odd that it seems that running webpack manually works, but running with jupyter lab build doesn’t. It may have to do with how we try to automatically do memory management for old node versions (will be gone in jlab 4, and maybe in jlab 3).
What if you do this to set the space node uses to 3GB (it defaults to 4GB, perhaps that is where it is running into issues)
It does not work either but I got the memory error with the “direct” method when I added multiple extensions (not so many, unfortunately… mainly because of some core assets being huge compared to the recommended limit ) :
webpack --config webpack.prod.minimize.config.js
<--- Last few GCs --->
[2246:0x4640bc0] 170951 ms: Mark-sweep 756.0 (867.1) -> 744.1 (839.6) MB, 600.6 / 0.0 ms (average mu = 0.898, current mu = 0.771) allocation failure scavenge might not succeed
<--- JS stacktrace --->
FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
Abandon
And I was able to overcome it with following command :
which means that you are right on both your affirmations, I noticed that the error at Object.<anonymous> appear when webpack is not started from /usr/local/share/jupyter/lab/staging and do not find the file index.js in case it helps to find a solution.
Many thanks @jasongrout I’ll test it ASAP and send feedback, at the moment I have to work on the raspi Os releases with jupyterlab not sending warning messages included.
Sorry @jasongrout to come back so late. Here are some news after jlab3 upgrade or full install :
the installation or upgrade works perfectly
lab build works perfectly with raspberry Pi4 outside lab
balfourdowns-€ : sudo systemctl stop jupyterlabhub
vidal[~] balfourdowns-€ : sudo jupyter lab build
[LabBuildApp] JupyterLab 3.0.5
[LabBuildApp] Building in /usr/local/share/jupyter/lab
[LabBuildApp] Building jupyterlab assets (production, minimized)
from inside lab sometimes build fails but that may be caused by bad sources in the extension menu (some cleaning in there might be very usefull )
Sstandard build still fails with raspberry3 with the memory heap bug . I know that JS is fantastic but it is also evil; there is a new version every second day, to compile it steals all your memory and your neighbour’s also, nobody cares about backwards compatibility… But a fix is necessary for our low cost, low energy, low memory tools.
I have released binary debian images with pre-compiled preinstalled jupyterhub lab service started at boot. If you are the lucky owner of a raspi or know one who has got one; you may test from one of the images provided here. I am preparing a page in english before publicising it.
Here is the output from lab build with webpack and the above trick
NODE_OPTIONS="--max-old-space-size=850" webpack --config webpack.prod.minimize.config.js
(node:3524) [DEP_WEBPACK_MAIN_TEMPLATE_GET_ASSET_PATH] DeprecationWarning: MainTemplate.getAssetPath is deprecated (use Compilation.getAssetPath instead)
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:3524) [DEP_WEBPACK_MAIN_TEMPLATE_GET_PUBLIC_PATH] DeprecationWarning: MainTemplate.getPublicPath is deprecated (use Compilation.getAssetPath(compilation.outputOptions.publicPath, options) instead)
(node:3524) [DEP_WEBPACK_COMPILATION_ASSETS] DeprecationWarning: Compilation.assets will be frozen in future, all modifications are deprecated.
BREAKING CHANGE: No more changes should happen to Compilation.assets after sealing the Compilation.
Do changes to assets earlier, e. g. in Compilation.hooks.processAssets.
Make sure to select an appropriate stage from Compilation.PROCESS_ASSETS_STAGE_*.
(node:3524) [DEP_WEBPACK_MODULE_ISSUER] DeprecationWarning: Module.issuer: Use new ModuleGraph API
assets by path *.js 7.23 MiB
assets by chunk 4.56 MiB (id hint: vendors)
assets by status 3.47 MiB [big] 8 assets
18 assets
assets by chunk 2.81 MiB (auxiliary id hint: vendors)
assets by path *.svg 1.73 MiB 3 assets
assets by path *.eot 367 KiB 3 assets
assets by path *.ttf 366 KiB 3 assets
assets by path *.woff 208 KiB 3 assets
assets by path *.woff2 168 KiB
asset 3ceb50e7bcafb577367c21b53064e00c.woff2 78.4 KiB [emitted] (auxiliary id hint: vendors)
asset c967a94cfbe2b06627ffe7861c741a61.woff2 76.6 KiB [emitted] (auxiliary id hint: vendors)
asset 1008b5226941c24f44681061d347ad5e.woff2 13.2 KiB [emitted] (auxiliary id hint: vendors)
asset index.html 1.53 KiB [emitted]
orphan modules 4.67 MiB [orphan] 1233 modules
runtime modules 97.8 KiB 14 modules
built modules 13.3 MiB (javascript) 4.43 KiB (share-init) 4.43 KiB (consume-shared) [built]
modules by path ./node_modules/ 13.3 MiB
cacheable modules 13.3 MiB 977 modules
3 modules
provide-module modules 4.43 KiB 108 modules
consume-shared-module modules 4.43 KiB
modules by path consume shared module (default) @jupyterlab/ 3.53 KiB 41 modules
modules by path consume shared module (default) @lumino/ 588 bytes 14 modules
modules by path ./build/ 22.7 KiB
./build/bootstrap.js 3.78 KiB [built] [code generated]
./build/index.out.js + 1 modules 18.9 KiB [built] [code generated]
4 modules
WARNING in ./node_modules/vega-lite/build/src/index.js 5:0-30
Should not import the named export 'version' (reexported as 'version') from default-exporting module (only default export is available soon)
@ consume shared module (default) vega-lite@^4.0.2 (strict) (fallback: ./node_modules/vega-lite/build/src/index.js)
@ ./node_modules/vega-embed/build/vega-embed.module.js 6:0-44 189:15-29
@ consume shared module (default) vega-embed@^6.2.1 (strict) (fallback: ./node_modules/vega-embed/build/vega-embed.module.js)
@ ./node_modules/@jupyterlab/vega5-extension/lib/index.js 166:28-48
@ consume shared module (default) @jupyterlab/vega5-extension@~3.0.2 (strict) (fallback: ./node_modules/@jupyterlab/vega5-extension/lib/index.js)
@ ./build/index.out.js 153:16-54
@ ./build/bootstrap.js
WARNING in vega-expression
Multiple versions of vega-expression found:
3.0.1 ./~/vega-lite/~/vega-expression from ./~/vega-lite/build/src/compile/data/expressions.js
4.0.1 ./~/vega-expression from ./~/vega-functions/build/vega-functions.module.js
Check how you can resolve duplicate packages:
https://github.com/darrenscerri/duplicate-package-checker-webpack-plugin#resolving-duplicate-packages-in-your-bundle
WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets:
e33e2cf6e02cac2ccb77689015f11c56.svg (730 KiB)
371dbce0dd46bd4d2033eb2334f28580.svg (896 KiB)
9741.b5e38b048132c3b57b8b.js (511 KiB)
25.282b82e1d552316d115a.js (303 KiB)
jlab_core.690a398af0cc8e4bdaeb.js (1.96 MiB)
714.286e0fa39e35ace9082c.js (292 KiB)
1281.d6b5b4c4adb657ccc9a7.js (450 KiB)
807.b65d521f16651a48f0eb.js (744 KiB)
9473.ded9e7826e2dfdaf9757.js (315 KiB)
2719.28123e57f2abeb3891b7.js (687 KiB)
2156.da63a4f0e2cc20860144.js (253 KiB)
webpack 5.13.0 compiled with 3 warnings in 522775 ms
asset index.css 16.1 KiB [compared for emit] (name: index)
asset index.js 0 bytes [compared for emit] [minimized] (name: index)
Entrypoint index 16.1 KiB = index.css 16.1 KiB index.js 0 bytes
./node_modules/@jupyterlab/theme-dark-extension/style/index.css 50 bytes [built] [code generated]
css ./node_modules/css-loader/dist/cjs.js!./node_modules/@jupyterlab/theme-dark-extension/style/variables.css 15.3 KiB [code generated]
css ./node_modules/css-loader/dist/cjs.js!./node_modules/@jupyterlab/theme-dark-extension/style/base.css 472 bytes [code generated]
css ./node_modules/css-loader/dist/cjs.js!./node_modules/@jupyterlab/theme-dark-extension/style/index.css 342 bytes [code generated]
webpack 5.13.0 compiled successfully in 25038 ms
asset index.css 15 KiB [compared for emit] (name: index)
asset index.js 0 bytes [compared for emit] [minimized] (name: index)
Entrypoint index 15 KiB = index.css 15 KiB index.js 0 bytes
./node_modules/@jupyterlab/theme-light-extension/style/index.css 50 bytes [built] [code generated]
css ./node_modules/css-loader/dist/cjs.js!./node_modules/@jupyterlab/theme-light-extension/style/variables.css 14.2 KiB [code generated]
css ./node_modules/css-loader/dist/cjs.js!./node_modules/@jupyterlab/theme-light-extension/style/base.css 472 bytes [code generated]
css ./node_modules/css-loader/dist/cjs.js!./node_modules/@jupyterlab/theme-light-extension/style/index.css 342 bytes [code generated]
webpack 5.13.0 compiled successfully in 24982 ms
Thanks for your help and congrats for jlab3 it is really top !
)
with the memory heap bug 
. I know that JS is fantastic but it is also evil; there is a new version every second day, to compile it steals all your memory and your neighbour’s also, nobody cares about backwards compatibility… But a fix is necessary for our low cost, low energy, low memory tools.