Sorry for the delay in replying!
open discussions
There’s a slot on Tuesday called Rethinking Notebook Cells (GitHub issue) on the Community Calendar… though attendance has been pretty sparse, recently (mostly the host and myself). But more’s the merrier!
There are number of ideas in the Jupyter Enhancement Proposal pipeline that would be very useful to effecting real change, mostly around standards alignment… of note, one just got merged (with work not yet started) which would create a schemas.jupyter.org (or something) which would serve as both a long-term, resolvable home for things, and documentation. The “or something” is partially driven by a desire for it to be able to host a number of specs, including grammars and derived type stubs, so that some of the ideas can be realized more concretely without every implementation/language starting from scratch.
However, there are some other proposals that would move Jupyter further away from markdown “standards” to more platform-specific features, and introducing even more “special” schema features per cell type, etc.
“literate programming”
Anyhow: part of the novel challenge is evolving the classic tangle/weave phases of literate programming production to something which takes into account not only the many languages supported by Jupyter tools, but also the conflict between the “high energy” state of authoring a Jupyter document and the “low energy” state of viewing a static(-ish), archival-grade state… in a sense this even goes beyond the standard concept of “hypermedia,” as an author/viewer would likely expect to see per-keystroke/mouse nudge changes reflected in one, or multiple, places in a document… or even multiple documents. Of course, this immediately crashes into the The Hard Problems of naming things and cache invalidation.
long-term solutions
If those are “solved”, I’ve imagined repurposing the markdown link syntax, already overloaded at the protocol level for markdown cell attachments, e.g.
The value of `x` was .
Which would be naively rendered with a “broken” image (showing the alt text):
The value of
xwas.
By moving transclusion out of the kernel language, no particular language/format would be preferred, and no new syntax would be created. In its “low energy” state, the resulting HTML would be fully resolved as an output (rather than the “special” state of markdown). In the “high energy” state, DOM could be updated dynamically, with anything that knows how to work with rich displays (usually one of the first features implemented in a kernel). It gets fun when considering the ready-built ecosystem of Jupyter Widgets, some of which would even work without kernels (though, generally, with CDN, buh), such that was above could be replaced with is now.
By adopting “real” URL/URIs, transclusion could theoretically be extended to embedding cells, or whole other notebooks, inside other cells. The other knock-on effect of strong, standards-based IDs and references is the improved feasibility of oft-requested features like annotation and single- or multiple-document internationalization.