What is installed by default with Binder?

Binder
1

I understand that Binder build the environment specified in my repo (e.g., for Python, using requirements.txt) but even with an empty specification, there’s a lot already in the Binder environment. The necessaries for Jupyter Lab, of course, but a lot more too it seems. E.g. I can import ipywidgets without error.

What Python packages are installed by Binder by default (e.g., what is its requirements.txt); and what additional Jupyter Lab configuration is preformed by default (e.g., what jupyter labextension install ... etc. are preformed)?

(Noob)

2

You’ll want to peruse repo2docker codebase. For example, if you only provide a requirements.txt, you’ll get

  • (old) nodejs, less and unzip from apt
  • python 3.7 installed with python which resolves to this conda-lock-compatible list of packages
    • this includes
      • jupyterlab_widgets (which provides @jupyter-widgets/*)
      • jupyterlab-offline-notebook (which provides its eponymous package)

Once you start asking for r, julia, etc. it gets… more complicated, with some packages installed from… all over. r, in particular, causes a cascading download of rstudio and a bunch of other stuff.

As of JupyterLab 3.0, there is no specific jupyterlab handling performed, as everything is handled by the pre-built extensions installed by conda (or by your requirements.txt).

From a performance perspective, you should make every effort not to use “source” extensions via jupyter lab build or jupyter labextension install in postBuild, as this will usually a) increase your build time substantially and b) reduce the reproducibility. Of course, if you have extensions you need that haven’t updated, you’ll still have to take the hit, and download 1gb+ of node_modules, run webpack, etc. then have binder push all of that trash to the various nodes.

However, if you/your users don’t use/like the giant text labels added to the notebook toolbar by jupyterlab-offline-notebook it’s “safe” (won’t invoke npm/webpack) to do jupyter labextension disable jupyterlab-offline-notebook.

Additionally, to enable jupyterlab 3.1’s collaborative features, you’ll have to do a little bit of extra work:

  • in requirements.txt, specify jupyterlab >=3.1 (or a specific version)
    • probably install jupyterlab-link-share to avoid having to root around for your token
  • in jupyter_config.json specify {"LabApp": {"collaborative": true}}
    • using this file (instead of jupyter_notebook_config.json and/or jupyter_server_config.json) avoids having to think too much about launching jupyter notebook (the default) vs jupyter lab or jupyter server

For a more concrete example, see this gist (and just not use rc2).

4 Likes
3

If I want a 3.1 (or later) can I get that another way (without polluting the requirements.txt), e.g., with some kind of build hook?

4

FYI there’s a PR open to upgrade lab to 3.1 Bump environment by manics · Pull Request #1069 · jupyterhub/repo2docker · GitHub (though that probably needs bumping to 3.1.1)

1 Like
5

pip is only semi-repeatable between multiple invocations (e.g somepackage[extras] may get ignored on successive solves), so to the extent you can, you’ll want everything in one (set of) requirements.txt installed by one pip install. Same goes for conda (well, mamba) installs of environment.yml.

You can create a binder-specific configuration in a binder/ (or .binder/, which by default will be hidden in the file tree). This allows you to specify other fun demo toys… some good ones are:

  • the aforementioned jupyterlab-link-share
  • nbgitpuller, which is useful if your environment and code change less than the content you want in your demo
    • this works by URL parameters, and the links can get pretty long, but can be generated
  • jupyter-videochat for voice/video/screensharing (disclaimer: maintainer)
    • can also open a chat by URL parameter

You can even pre-populate multiple Lab tabs that will be open by default.

So, putting these all together, you might have something like:

/
  .git/
  requirements.txt                      # your _actual_ dependencies, ignored...
  .binder/
    requirements.txt                    # this runs first...
       -r ../requirements.txt           # ...bring in whatever's there
       -e ..                            #... if you've got a setup.py
       nbgitpuller
       jupyterlab-link-share
       jupyter-videochat 
    postBuild                           # then this...
       #!/usr/bin/env bash                  
       #   though you can specify anything, 
       #   e.g. #!/usr/bin/env python
       cp .binder/jupyter_config.json .
       jupyter lab workspaces import .binder/my-demo.jupyterlab-workspace
    jupyter_config.json                 # finally, this is used at runtime
    my-demo.jupyterlab-workspace

Though if you do all that, your link wil be super-duper long, and you’ll probably want to make a bit.ly out of it or something.

2 Likes
6

So if I’ve got this right, .binder/requirements.txt is cool: it basically lets me specify any additional packages I want for use specifically with Binder, w/o having to touch my requirements.txt. Is there (naive question) a way to do the equivalent with Jupyter Lab locally?

7

Everything should work locally.

Assuming a posix environment, on an Intel CPU, from the root of your repo:

pip install -r ./.binder/requirements.txt
bash ./.binder/postBuild

If course, you won’t have all the stuff you get from repo2docker.

If you’re on windows or an apple m1… I can only recommend trying mambaforge, and getting cozy with per-project, isolated environments, or making darned sure you have the exact proprietary compiler your most complicated package needs.

1 Like
8

If I’ve got a .binder/, I’ve got to put my runtime.txt there, rather than at the root, right?

9

This is nice, but is there a way to do this with jupyterlab-git?

10

right, .binder transitively ignores binder and .

If you PR it, they will come. :baseball: Maybe.

1 Like
11

Beyond my skills. But I can ask.