Is there anything new on Pyodide?

Pyodide is a way to run Python “natively” in a browser via web assembly (wasm). About two years ago, @bollwyvl threw together some prototype work on a Pyodide-based Jupyter kernel called jyve. That was forked and extended to be a Jupyter labextension (I’m not sure if the author is on this discourse), also done about two years ago. Jyve is even mentioned in a May 2019 open ticket in the Pyodide issues.

I am also aware that Iodide exists, not to be conflated with Pyodide although they’re written by the same team (git / blog post / demo). While Iodide does offer a Jupyter-esque interface, it is not Jupyter in a browser.

To the topic of this post: is there anything else going on in the Jupyter/Pyodide realm? The best answer would be, “yes you can serve a static page (with wasm) that spins up a Jupyter single-user instance running in Pyodide, a Pyodide-based kernel, and Notebooks ‘just work’ to include rendering input() and ipywidgets.” I don’t expect that exists, or it would have absolutely made the rounds. But is there anything between the jyve prototyping two years ago and now that we should look at as “prior art”?

Ultimately, the use-case here is having another infrastructure option for executing Jupyter Notebooks – in this case needing only the browser itself and not even a Binder or similar corporate infrastructure solution. I’ll acknowledge ahead of time that there are a myriad of other challenges to address running a Notebook designed in a traditional Jupyter environment in Pyodide, such as http requests/networking needing to go through the the javascript api instead of requests/sockets.

Thanks,
-Kafonek

4 Likes

Howdy, thanks for bringing this up!

Still very bullish on “click button, get github-pages-hostable site with kernels, labextensions, notebooks and data”! Maybe we’ll be able to give binder a day off now and again. The other side of it, e.g. live-hacking and inspecting a running Jupyter client, is of course also quite compelling, and browser-based kernels and language servers would make that a lot nicer.

In the meanwhile, there has been some additional promising work from the @stdlib-js folks on jupyter-stdlib-browser-kernel folks and @jtpio on p5.js. Further, it seems every couple weeks another WASM-based language runtime pops up. It could well be that wasm might even have something to say on the server side, as tools like wasmer and wasmtime get closer to an actual emcripten-once-run-anywhere deployment scenario, which may eventually have a stronger untrusted code execution story than containers.

For the record, I never actually got pyodide working satisfactorily in jyve, fully, under continuous integration against lab 0.3x, and the approach was rather messy anyhow. I ran into enough issues porting to 1.0 (even statically-typed monkeypatches are still just patches) to give up, and have been too busy to attempt for 2.0. However, in the post-JupyterLab 3.0 world, where webpack is a build-time-only concern, the prospects are very bright indeed. I should really get back on the saddle and try to get the extension points jyve would need to “for real” be part of the stack: basically a PR to @jupyterlab/services to add a way to register more kernelspecs, and then be able to fulfill them in a kernel-message-spec respecting way.

On the emscripten build side of pyodide: they’re doing some heroic stuff, but the long con probably needs a conda-forge-like, multi-repo approach, ideally in conda-forge itself.

Once all these titanic builds exist and are installable via “normal means”, being able to download them “out of band” (e.g. via pip/conda, then do a simple copy-and-archive job to make it re-hostable) is important. But the prospect of being able to handle the distribution of these in a peer-to-peer many (via IPFS or dat or webtorrents) may well become very desirable, especially once the data sets one might wish to work with start to be stressful to the browser. Whatever happens, it’s going to be slow, and so WebWorker/ServiceWorker probably need to get into the architecture sooner, rather than later.

Cheers!

4 Likes

Made this issue to discuss:

1 Like