JupyterLab extension Node polyfill and Buffer error

JupyterLab extensions
, ,
1

Hello.

I am building a JupyterLab extension that uses quite a lot of large Node modules. Currently, I am trying to render my main-menu extension but am having trouble with it when JupyterLab is running, so I wanted to ask for help if anyone has some experience with the problem I am facing.

My installation went like this:

pip install -e . - without error
jlpm - without error
jlpm run build - with warnings
jupyter labextension install . - with errors, but was able to manually solve them (polyfill issues for http, https, stream)

When I run jupyter lab, the extension is shown in the menu bar, but clicking on it doesn’t render the commands. The console has the following error:

ReferenceError: Buffer is not defined at varintEncode (util.js:33:1) at ./node_modules/multicodec/src/varint-table.js (varint-table.js:11:1) at __webpack_require__ (bootstrap:19:1) at ./node_modules/multicodec/src/index.js (index.js:16:31) at __webpack_require__ (bootstrap:19:1) at ./node_modules/content-hash/src/index.js (index.js:19:16) at __webpack_require__ (bootstrap:19:1) at ./node_modules/web3-eth-ens/lib/lib/contentHash.js (contentHash.js:31:19) at __webpack_require__ (bootstrap:19:1) at ./node_modules/web3-eth-ens/lib/ENS.js (ENS.js:26:19)

None of these node_modules are explicitly in my package.json, they are dependencies from other node_modules I am using. My webpack.config.js looks like this:

const NodePolyfillPlugin = require("node-polyfill-webpack-plugin");
const webpack = require("webpack");

module.exports = {
  stats: {
    errorDetails: true,
    children: true,
  },
  plugins: [
    new NodePolyfillPlugin(),
    // Work around for Buffer is undefined:
    // https://github.com/webpack/changelog-v5/issues/10
    new webpack.ProvidePlugin({
      Buffer: ["buffer", "Buffer"],
    }),
    new webpack.ProvidePlugin({
      process: "process/browser",
    }),
  ],
  resolve: {
    extensions: [".tsx", ".ts", ".js", ".jsx", ".css"],
    modules: ["node_modules"],
    fallback: {
      http: require.resolve("stream-http"),
      https: require.resolve("https-browserify"),
      stream: require.resolve("stream-browserify"),
      buffer: require.resolve("buffer"),
      assert: require.resolve("assert/"),
      // multicodec: false, -> tried these, didn't work
      // "content-hash": false,
      // "web3-eth-ens": false
      // "assert": false,
      // "browser": false,
    },
  },
};

and in my package.json, I also have the following browser config:

"browser": {
    "stream": "stream-browserify",
    "http": "stream-http",
    "https": "https-browserify",
    "buffer": "buffer",
    "web3": false,
    "fs": false
  },

I was recommended to configure webpack in the jupyterlab gitter channel, but I am not sure what exactly I need to configure, because I tried adding the explicit node_modules that came up in the Buffer error, but that didn’t make a difference. I appreciate any suggestions.

2

Just a sanity check: do you have webpackConfig in your package.json to let JupyterLab know that your extension wants to modify webpack config? See Extension Developer Guide — JupyterLab 3.4.4 documentation

3

Yes, I have "webpackConfig": "./webpack.config.js", in my package.json in the "jupyterlab" config.