Is this dirty build hack acceptable to use extensions on raspberry?

I am trying to use jupyterlab hub on raspberries. I have seen in the logs that the “build memory error” derived from a problem with webpack. It is impossible to build even when there is no extension. I have succeeded with a dirty trick, is it acceptable ? would it be possible to build jupyterlab that way properly ?

I have installed command line webpack webpack-cli webpack-bundle-analyzer, change directory to /usr/local/share/jupyter/lab/staging then execute :

webpack --config webpack.prod.minimize.config.js

which worked properly (or almost) yielding some informative messages on the assets size and enabling jupyterlab without error message and with available extensions :

     WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
    This can impact web performance.
    Assets: 
      664de3932dd6291b4b8a8c0ddbcb4c61.svg (896 KiB)
      2f12242375edd68e9013ecfb59c672e9.svg (730 KiB)
      vendors~main.825fba72abae7de1679c.js (4.69 MiB)
      2.9ccb7977a199a4a50689.js (454 KiB)
      3.50663db94d152dab6429.js (883 KiB)
      4.69406a7d6939b46f4e98.js (253 KiB)
    
    WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
    Entrypoints:
      main (4.71 MiB)
          vendors~main.825fba72abae7de1679c.js
          main.4363983affea8feaa5a7.js

You can run jupyter lab build without any extensions, and it will just do essentially what you did manually with webpack. That’s a fully supported feature.

The standard command jupyter lab build aways fails even with pi4-4Gb without dev and without minimize, with the following error message :

 [LabBuildApp] JupyterLab 2.2.9
[LabBuildApp] Building in /usr/local/share/jupyter/lab
[LabBuildApp] Building jupyterlab assets (build:prod:minimize)
Build failed.
.../...

With explanation in the log :

Error: Command failed: /usr/local/share/jupyter/lab/staging/node_modules/.bin/webpack --config webpack.prod.minimize.config.js
    at checkExecSyncError (node:child_process:636:11)
    at Object.execFileSync (node:child_process:654:15)
    at Object.<anonymous> (/usr/local/share/jupyter/lab/staging/node_modules/@jupyterlab/buildutils/lib/ensure-max-old-space.js:38:17)
    at Module._compile (node:internal/modules/cjs/loader:1083:30)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1112:10)
    at Module.load (node:internal/modules/cjs/loader:948:32)
    at Function.Module._load (node:internal/modules/cjs/loader:789:14)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:72:12)
    at node:internal/main/run_main_module:17:47 {
  status: null,
  signal: 'SIGILL',
  output: [ null, null, null ],
  pid: 1572,
  stdout: null,
  stderr: null
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[LabBuildApp] JupyterLab failed to build
[LabBuildApp] Traceback (most recent call last):
[LabBuildApp]   File "/usr/local/lib/python3.7/dist-packages/jupyterlab/debuglog.py", line 47, in debug_logging
    yield
[LabBuildApp]   File "/usr/local/lib/python3.7/dist-packages/jupyterlab/labapp.py", line 138, in start
    raise e
[LabBuildApp]   File "/usr/local/lib/python3.7/dist-packages/jupyterlab/labapp.py", line 135, in start
    command=command, app_options=app_options)
[LabBuildApp]   File "/usr/local/lib/python3.7/dist-packages/jupyterlab/commands.py", line 460, in build
    command=command, clean_staging=clean_staging)
[LabBuildApp]   File "/usr/local/lib/python3.7/dist-packages/jupyterlab/commands.py", line 661, in build
    raise RuntimeError(msg)
[LabBuildApp] RuntimeError: JupyterLab failed to build
[LabBuildApp] Exiting application: JupyterLab

Am I doing something wrong ?

Thanks

Does jupyter lab build --minimize=False work? It may be that you run out of memory.

It is a little odd that it seems that running webpack manually works, but running with jupyter lab build doesn’t. It may have to do with how we try to automatically do memory management for old node versions (will be gone in jlab 4, and maybe in jlab 3).

What if you do this to set the space node uses to 3GB (it defaults to 4GB, perhaps that is where it is running into issues)

NODE_OPTIONS=--max-old-space-size=3000 jupyter lab build

I just submitted https://github.com/jupyterlab/jupyterlab/pull/9282, which hopefully solves this problem for jlab 3.

1 Like

No it fails with the same error even if there is no extension.

It does not work either but I got the memory error with the “direct” method when I added multiple extensions (not so many, unfortunately… mainly because of some core assets being huge compared to the recommended limit ) :

webpack --config webpack.prod.minimize.config.js
<--- Last few GCs --->
[2246:0x4640bc0]   170951 ms: Mark-sweep 756.0 (867.1) -> 744.1 (839.6) MB, 600.6 / 0.0 ms  (average mu = 0.898, current mu = 0.771) allocation failure scavenge might not succeed
<--- JS stacktrace --->
FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
Abandon

And I was able to overcome it with following command :

NODE_OPTIONS="--max-old-space-size=3072" webpack --config webpack.prod.minimize.config.js

which means that you are right on both your affirmations, I noticed that the error at Object.<anonymous> appear when webpack is not started from /usr/local/share/jupyter/lab/staging and do not find the file index.js in case it helps to find a solution.

Many thanks @jasongrout I’ll test it ASAP and send feedback, at the moment I have to work on the raspi Os releases with jupyterlab not sending warning messages included.