Best practices for installing Jupyter nbextensions via pip

Meta
1

Hi, apologies if this isn’t the right forum - I know discourse.jupyter.org is mostly for JupyterHub and Binder issues, but please point me in the right direction if there’s a better place for Jupyter and especially nbextension development.

I want to distribute my Jupyter nbextension package, let’s call it foobar, with a simple pip install:

pip install foobar

My package includes an index.js file and notebook.json for Jupyter nbextension deployment. I’ve been able to get these included with python setup.py wheel as follows:

packages=find_packages('src'),
package_dir={'': 'src'},
py_modules=[splitext(basename(path))[0] for path in glob('src/*.py')],
include_package_data=True,  # Includes everything in src/

Also, the .js/.json files are included in src/foobar. These need to be enabled post-install.

I’ve also tried following the guide here:
https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Distributing%20Jupyter%20Extensions%20as%20Python%20Packages.html

However, this recommends using data_files, which Ionel Mărieș says to avoid like the plague:
https://blog.ionelmc.ro/presentations/packaging/#slide:16

Can anyone recommend best practices around this? I’d like to have the package pip install without sudo, and cross-platform ideally. However, it seems like the Jupyter link forces you to use --sys-prefix, which I’m not sure will always work, whereas --user should have no permissions issues (in theory).

Or, if not using data_files, is there any way to enable the notebook extensions as part of pip install, without the user requiring an extra step?

Thanks!

Vaughn

2

I’d say the avoidance of data_files is outdated advice, but it is based on a technically-true fact: Python doesn’t record where it put data files, which is frustrating, and installation schemes are technically allowed to put them wherever they want. The note about eggs is not at all relevant in 2018.

That said, if you are using any of these combinations:

  • system/standard python installs on mac, windows, or most linux/bsd/etc.
  • virtual envs on all platforms
  • conda envs on all platforms

which cover very close to all realistic scenarios, data_files work just fine.

Taking the “data_files almost always work” combined with “data_files don’t always work”, I would consider this the best practice:

  • rely exclusively on data_files for enable-at-install-time
  • provide a runtime install command (e.g. make sure jupyter nbextension install --py mypackage works) to recover from the rare cases where the data_files don’t work (or if users have done jupyter nbextension remove ... and want to reinstall). This means shipping the javascript sources in both package_data and data_files in a wheel. Your setup.py could copy the package_data to data_files.

Given how often data_files does work, I would say that it’s not a reasonable expectation for any of the peculiar Python installs to work without the extra explicit step.

1 Like
3

One thing to note with extensions requiring dependent packages is that they seem very brittle on Azure Notebooks eg https://github.com/akaihola/jupyter_abc/issues/2 (not sure if the underlying cause, or best practice, has been resolved yet?)

–tony