Miscellaneous questions on the usage of JupyterLite

General
1

Hello,

I have some basic usage questions about JupyterLite.

I’ve followed the nice Deploy your first JupyterLite website on GitHub Pages doc which proposes to use the jupyterlite/demo repo. This was indeed easy enough and brought me back in July:

One thing I really appreciated about this approach (i.e. template repository) is that I was able to get JupyterLite deployed on my GitHub Pages without any previous knowledge of GitHub Pages nor GitHub Actions. So thanks a lot for creating this template repo.

The only logical downside of not learning those technologies is that it’s a bit harder to follow-up on more complicated things (like following the Configure a JupyterLite site section)…

Updating JupyterLite

My copy of the jupyterlite/demo template got me version JupyterLite version 0.1.0-beta.10 while a beta 12 has been released since. Looking at the commits on jupyterlite/demo, is it true that I only need to update my requirements.txt file? Should I do it manully or using some tool to check the consistency?

Also, should I check and update the configuration of Github Actions?

EDIT: I juste read more carefully Deploy your first JupyterLite website on GitHub Pages, and it clearly mentions that updating JupyterLite should be done by changing requirements.txt. However the question of the consistency of manual edits still holds.

Using Micropip inside a script

Since there is no way to pre-run Python code (from what I got from open issues #461 and #508), I’ve gathered the setup code of my demo inside a script (microgrid_showcase_setup.py) which gets run inside the notebook using the %run -i magic (with -i to be able to change some global parameters later on).

It contains these package installation and import section:

try: # Install microgrids & ipywidgets packages in JupyterLite (if run in JupyterLite)
    import piplite
    async def install_packages():
        await piplite.install(['microgrids', 'ipywidgets'])
        print('Microgrids & ipywidgets packages installed with piplite.')

    # Get away with the fact that `await` is not allowed in a script:
    import asyncio
    loop = asyncio.get_event_loop()
    loop.run_until_complete(install_packages())
except ImportError:
    pass

from ipywidgets import interactive, fixed

However, when I run it for the first time, I get the ModuleNotFoundError: No module named 'ipywidgets' error. I only works if I re-run the cell (with %run -i my_setup.py) a second time (actually, this was the behavior in late July, but today it never works!).

Did I got someting wrong about using Micropip?

Installing ipywidgets fails (but it worked in late July!)

In a longer demo (https://microgrids-x.github.io/Microgrids.web/lab?path=Microgrid_py_PV_BT_DG.ipynb), I’m not using a setup script but rather running the package installation in a cell:

try:
    import piplite
    await piplite.install(['ipywidgets'])
except ImportError:
    pass

This was working in late July, but know I get ValueError: Couldn’t find a pure Python 3 wheel for ‘widgetsnbextension~=4.0’. You can use micropip.install(..., keep_going=True) to get a list of all packages with missing wheels.

Did I miss something? Is there an interference between a recent release of Jupyter widgets and my requirements.txt which contains ipywidgets>=7.7,<8 ?

1 Like
2

At some point we just need to make a %pip install magic work, but it will take some nasty rewriting to avoid hitting asyncio issues: in the interim, I guess don’t try to be too clever with asyncio tricks… run_until_complete generally won’t work due to hacks in both the ipython and pyodide stacks.

Regarding widgets: we’re working on an interim approach to supporting ipywidgets 8 and 7 in the same jupyterlite package by shipping two versions of the shim packages:

But even still, a site would need to ensure its labextension versions line up properly with the packages pip will bring in, and a requirements.txt that pins it, with piplite.install(['ipywidgets<7']). This is usually handled correctly if the default environment-sniffing is used.

Generally: once you get off the happy path of demos, it pays to do some of the more aggressive pinning techniques described in the jupyterlite docs… at some point, a pdm or pip-tools tool would likely make the most sense… while micropip has some newer features around freeze, they have other limitations (such as single versioning) that are somewhat challenging. And mainline pip… doesn’t really care about this use case.

1 Like
3

Thanks for the feedback. The situation seems more complicated than I expected.

In the mean time, I have updated my requirements.txt file to jupyterlite==0.1.0b12, taking jupyterlite/demo requirements as an example. However, I believe I must be missing something, because now even in a blank notebook (and after erasing website data from Firefox), I cannot install ipywidgets!

I’ve tried both:

  • await piplite.install(['ipywidgets==7']) (which yields ValueError: Can't find a pure Python 3 wheel for 'widgetsnbextension~=3.0.0'.) and
  • await piplite.install(['ipywidgets==8']) (equivalent to await piplite.install(['ipywidgets'])) which yields ValueError: Can't find a pure Python 3 wheel for 'widgetsnbextension~=4.0'.

Now, I just tried the same in the deployment of the demo repository, https://jupyterlite.github.io/demo/lab/ and I get the same error.

I’ve also tried the official Jupyterlite demo (:warning: not to be confused with the demo repo!) https://jupyterlite.readthedocs.io/en/latest/_static/lab/ and here, the installation of ipywidgets works!

So, I’ve opened an issue: ipywidgets fails to install in jupyterlite/demo deployment · Issue #97 · jupyterlite/demo · GitHub.

1 Like
4

The == will not look for anything except exactly that (with any number of .0 suffixes), so:

await piplite.install(['ipywidgets==7'])

Is looking for 7.0.0.

On the the repo-based demo, ipywidgets<8 works fine.

the official Jupyterlite demo

Right, here we do many of the “offline” steps linked above… indeed, many of those were created because of that RTD-based demo deployment, which brings in quite a lot of widgets.

We pin down and check in the versions of the labextensions with the versions of the kernel packages, resulting in the two long lists of conda packages and wheels in jupyter_lite_config.json. Because pip-adjacent APIs are notoriously fickle, and are being overloaded at the pyodide, micropip and piplite layers, we don’t yet ship that capability, as there we have found too many edge cases to generalize the pattern.

As i mentioned, we’re working on getting another build of widgetsnbextension merged in before 0.1.0b13.

1 Like
5

OK, this was my poor understanding of your PR#783 (“await piplite.install('ipywidgets=8') should work”). Is this a typo =8 vs ==8? Indeed, the installation of ipywidgets<8 also works fine on my Microgrids.web demo, thanks!

So my only remaining issue is the need to run twice the setup script which contains the call to piplite.install (with what you called the “too clever trick with asyncio”). I’ll try adding a bit of delay (and as per Pyodide issue #2354, I cannot use time.sleep…)

6

I’ve tried adding a bit of sleep after the install, but the import still fails. It seems like loop.run_until_complete(install_packages) doesn’t block the execution flow.

try:
    import piplite
    import asyncio
    
    async def install_packages():
        await piplite.install(['microgrids', 'ipywidgets<8'])
        print('Microgrids & ipywidgets packages installed (with piplite).')
        await asyncio.sleep(0.5)

    # Get away with the fact that `await` is not allowed in a script:
    loop = asyncio.get_event_loop()
    loop.run_until_complete(install_packages())
except ImportError:
    pass

from ipywidgets import interactive, fixed
7

Is this a typo

yes.

loop.run_until_complete(install_packages) doesn’t block the execution flow.

Yes, this is the part that is unlikely to work today, and more sleep won’t do it, either.

Top-level await is supported in .ipy and .ipynb files run by modern ipykernel/IPython (any major versions still getting updates), so this is valid in a jupyterlite/jupyterlab setting:

try:
    import piplite
    await piplite.install(['microgrids', 'ipywidgets<8'])
finally:
    from ipywidgets import interactive, fixed
    import microgrids
1 Like