Source build of jupyterlab 2.2.x failed with settingregistry.ts:221:11 - error TS2722: Cannot invoke an object which is possibly 'undefined' (validate object)

JupyterLab
, ,
1

This could also re-titled as:

  • Need help in juypterlab 2.2.x source build behind corporate proxy/firewall using corporate artifactory registry/channels (in lieu of registry.yarnpkg.com)
  • I have been using conda and jupyterlab for quite some time. However, I am new to the jupyterlab source, and also new to npm/yarn/webpack/react. I have been reading up on these topics, and found many good resources in the web (big thanks to all the folks who took time to document the steps or provide answers to help us newbie to move along!)
  • I was able to make a build, using proxy to access registry.yarnpkg.com. However, when I tried to regenerate yarn.lock so that it can use artifactory registry/channels, the jlpm build step failed.
  • I don;t know enough about js/yarn to fix the issue. Help is greatly appreciated!
  • I wonder if it is the right idea to regenerate yarn.lock. (The alternative possibly to convert all reference of yarnpk.com to artifactory URL, but that URL is pretty ugly, and I am not sure if manually updating yarn.lock a good idea)

Some facts of the build:

  • The source code: zip file from jupyterlab 2.2.x commit in late Nov. (after 2.2.9 has been released)
  • The conda env was created: conda create jlab-dev python=3.6 notebook=6 nodejs=14 nb_conda_kernels
  • Build was done on a Mac, using .npmrc and .yarnrc, defining registry to be from artifactory, and http/https proxy. Strict-ssl set to false.
  • node --version # v14.15.1
  • yarn --version # 1.21.1

Build-1: Using the yarn.lock in the release works:

  • Build using Mac
  • Behind corporate proxy/firewall, use https proxy to access yarnpkg.com
  • using the released yarn.lock.
  • Follow the instructions in creating new conda env, pip install -e ., jlpm install, jlpm run build, jlpm build:core, jupyter lab build; all work OK
  • BTW, to get pip install -e . to work, I need to: export NODE_TLS_REJECT_UNAUTHORIZED=0

Build-2: Regenerate yarn.lock does not work:

  • Then in another fresh build, using a fresh conda env, with a newly unzip 2.2.x source.
  • This time, I remove the yarn.lock
  • Why remove yarn.lock? The file has reference to https://registry.yarnpkg.com/… I need to remove any direct reference to yarnpkg.com, and replace with the corporate artifactory channel so that I can do a CICD build.
  • From what I read, yarn.lock will be regenerated by yarn/jlpm from package.json, and will use the registry setting in my ~/.npmrc and ~/.yarnrc. However, the downside is the versions of npm/yarn packages are not locked, and may introduce build issue.
  • pip install -e . # worked OK
  • rm yarn.lock # remove yarn.lock
  • jlpm # ran OK, and generated yarn.lock with reference to artifactory (and no https://registry,yarnpkg.com). This step does generate warnings (see below)
  • jlpm install # worked OK
  • jlpm run build # failed with the following error msg:

jlpm run build yarn run v1.21.1
jlpm run build:dev jlpm run integrity && jlpm run build:packages && cd dev_mode && jlpm run build
node scripts/ensure-buildutils.js && node buildutils/lib/ensure-repo.js Repo integrity verified! cd packages/metapackage && jlpm run build
$ tsc -b

../settingregistry/src/settingregistry.ts:221:11 - error TS2722: Cannot invoke an object which is possibly 'undefined'.
221     if (!(validate(schema) as boolean)) {
              ~~~~~~~~
../settingregistry/src/settingregistry.ts:222:14 - error TS2532: Object is possibly 'undefined'.
222       return validate.errors as ISchemaValidator.IError[];
                 ~~~~~~~~
../../node_modules/@types/react/index.d.ts:2978:14 - error TS2300: Duplicate identifier 'LibraryManagedAttributes'.

2978         type LibraryManagedAttributes<C, P> = C extends React.MemoExoticComponent<infer T> | React.LazyExoticComponent<infer T>
                  ~~~~~~~~~~~~~~~~~~~~~~~~
  ../../node_modules/@types/react-dom/node_modules/@types/react/index.d.ts:2982:14
    2982         type LibraryManagedAttributes<C, P> = C extends React.MemoExoticComponent<infer T> | React.LazyExoticComponent<infer T>
                      ~~~~~~~~~~~~~~~~~~~~~~~~
    'LibraryManagedAttributes' was also declared here.
../../node_modules/@types/react-dom/node_modules/@types/react/index.d.ts:2982:14 - error TS2300: Duplicate identifier 'LibraryManagedAttributes'.
2982         type LibraryManagedAttributes<C, P> = C extends React.MemoExoticComponent<infer T> | React.LazyExoticComponent<infer T>
                  ~~~~~~~~~~~~~~~~~~~~~~~~
  ../../node_modules/@types/react/index.d.ts:2978:14
    2978         type LibraryManagedAttributes<C, P> = C extends React.MemoExoticComponent<infer T> | React.LazyExoticComponent<infer T>
                      ~~~~~~~~~~~~~~~~~~~~~~~~
    'LibraryManagedAttributes' was also declared here.

Found 4 errors.

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.

  • line 221 is inside _addSchema(), and line 218 defines: const validate = validator.getSchema(‘jupyterlab-plugin-schema’);

  • Not sure why validate is undefined… maybe due to jupyterlab-plugin-schema not found? (I wasn;t able to find this schema file in either Build-0 or Build-1 build trees, but this file is being used in several places. Is this file being generated during build process? )

  • The settingregistry.ts are the same between Build-1 and Build-2 (both version 2.2.5, 1167 lines).

  • For completeness, here are the warning message when running jlpm:

$ jlpm                                                                                                                                                                                              cchung@WM-C02WJ5XDHTD6
yarn install v1.21.1
info No lockfile found.
[1/4] 🔍  Resolving packages...
⡀ @typescript-eslint/eslint-plugin@^2.27.0(node:38297) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(Use `node --trace-warnings ...` to show where the warning was created)
warning lerna > @lerna/bootstrap > @lerna/symlink-dependencies > @lerna/create-symlink > @zkochan/cmd-shim > mkdirp-promise@5.0.1: This package is broken and no longer maintained. 'mkdirp' itself supports promises now, please switch to that.
warning lerna > @lerna/bootstrap > @lerna/run-lifecycle > npm-lifecycle > node-gyp > request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
warning lerna > @lerna/bootstrap > @lerna/run-lifecycle > npm-lifecycle > node-gyp > request > har-validator@5.1.5: this library is no longer supported
warning lerna > @lerna/create > globby > fast-glob > micromatch > snapdragon > source-map-resolve > resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
warning lerna > @lerna/create > globby > fast-glob > micromatch > snapdragon > source-map-resolve > urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
warning typedoc > highlight.js@9.18.5: Support has ended for 9.x series. Upgrade to @latest
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/application-extension > typedoc > highlight.js@9.18.5: Support has ended for 9.x series. Upgrade to @latest
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > typedoc > highlight.js@9.18.5: Support has ended for 9.x series. Upgrade to @latest
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/application-top > yarn-deduplicate@1.2.0: Breaks Node < 10 compatibility
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/services > text-encoding@0.7.0: no longer maintained
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/buildutils > crypto@1.0.1: This package is no longer supported. It's now a built-in Node module. If you've depended on crypto, you should switch to the one that's built-in.
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/testutils > lighthouse > mkdirp@0.5.1: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/ui-components > @blueprintjs/core > popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/ui-components > @blueprintjs/core > react-popper > popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/json-extension > react-json-tree > babel-runtime > core-js@2.6.12: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/ui-components > @storybook/addon-actions > @storybook/components > popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/testutils > lighthouse > chrome-launcher > mkdirp@0.5.1: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/testutils > lighthouse > jsonld > request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/application-top > webpack > watchpack > watchpack-chokidar2 > chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/testutils > lighthouse > mkdirp@0.5.1: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/ui-components > @blueprintjs/core > popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/ui-components > @blueprintjs/core > react-popper > popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/json-extension > react-json-tree > babel-runtime > core-js@2.6.12: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/ui-components > @storybook/addon-actions > @storybook/components > popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/testutils > lighthouse > chrome-launcher > mkdirp@0.5.1: Legacy versions of mkdirp are no longer supported. Please update to mkdirp 1.x. (Note that the API surface has changed to use Promises in 1.x.)
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/testutils > lighthouse > jsonld > request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/application-top > webpack > watchpack > watchpack-chokidar2 > chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/application-top > webpack > watchpack > watchpack-chokidar2 > chokidar > fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/ui-components > @storybook/react > react-dev-utils > fork-ts-checker-webpack-plugin > chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/ui-components > @storybook/addon-actions > @storybook/components > react-syntax-highlighter > highlight.js@9.13.1: Version no longer supported. Upgrade to @latest
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/ui-components > @storybook/addon-actions > @storybook/components > react-syntax-highlighter > lowlight > highlight.js@9.13.1: Version no longer supported. Upgrade to @latest
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/application > jest > @jest/core > jest-config > jest-environment-jsdom > jsdom > request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
warning workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/application > jest > @jest/core > jest-config > jest-environment-jsdom > jsdom > request-promise-native@1.0.9: request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
warning "@typescript-eslint/eslint-plugin > tsutils@3.17.1" has unmet peer dependency "typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta".
warning "lerna > @lerna/version > @lerna/github-client > @octokit/rest > @octokit/plugin-request-log@1.0.2" has unmet peer dependency "@octokit/core@>=3".
warning " > react-docgen-typescript-loader@3.7.2" has unmet peer dependency "typescript@*".
warning "react-docgen-typescript-loader > @webpack-contrib/schema-utils@1.0.0-beta.0" has unmet peer dependency "webpack@^3.0.0 || ^4.0.0".
warning "react-docgen-typescript-loader > react-docgen-typescript@1.20.5" has unmet peer dependency "typescript@>= 3.x".
warning "workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > typedoc@0.17.0-3" has unmet peer dependency "typescript@>=3.7".
warning " > @jupyterlab/ui-components@2.2.4" has unmet peer dependency "react@~16.9.0".
warning "workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/application-extension > @lumino/coreutils@1.5.3" has unmet peer dependency "crypto@1.0.1".
warning "workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/ui-components > babel-loader@8.2.2" has unmet peer dependency "webpack@>=2".
warning "workspace-aggregator-9e208a69-81d6-40d1-b0a3-1f9721bf02e9 > @jupyterlab/ui-components > @storybook/addon-actions > @storybook/api@5.3.21" has unmet peer dependency "regenerator-runtime@*".
[4/4] 🔨  Building fresh packages...
success Saved lockfile.
$ node scripts/ensure-buildutils.js
✨  Done in 318.89s.

Thanks in advance!

2

Here is a (ugly?) workaround I was able to do: in stead of regenerating yarn.lock, I actually convert all registry.yarnpkg.com to our artifactory URL. With this approach, I can make sure the versions of node modules remained identical to what was in the release. Then my build works!

BTW, there are at least three files need to be updated: - jupyterlab-2.2.x/yarn.lock, jupyterlab-2.2.x/jupyterlab/staging/yarn.lock.yarn.js.

I say this is an ugly workaround, as I was reading that ‘though shall not update yarn.lock’, as such file can be generated by yarn -l. If anyone knows a better way, I would like to know. Thanks!