# Inline variable insertion in markdown

**URL:** <https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525>\
**Category:** Notebook\
**Tags:** notebook, feature-idea, markdown\
**Created:** [August 27, 2021, 5:35pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525 "2021-08-27T17:35:23Z")\
**Posts on this page:** 20\
**Page:** 4

<div class="post-metadata">

**Author:** ![jasongrout](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/jasongrout/32/371_2.png) [@jasongrout](https://discourse.jupyter.org/u/jasongrout)\
**Post date:** [September 10, 2021, 2:32am UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/62 "2021-09-10T02:32:43Z")

</div>

> [@agoose77](#):
>
> I remember discussing the concept of “rich markdown” (concerning widgets) with @jasongrout a while back. We both (IIRC) expressed an interested in the ability to have a WYSIWYG kind of editor for widgets-in-text.

Here is a tech demo I made years ago exploring some possibilities around rich javascript input in a code cell: [jsfiddle of codemirror live widgets · GitHub](http://bl.ocks.org/jasongrout/5378313)

---

<div class="post-metadata">

**Author:** ![fperez](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/fperez/32/148_2.png) [@fperez](https://discourse.jupyter.org/u/fperez)\
**Post date:** [September 10, 2021, 4:05am UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/63 "2021-09-10T04:05:59Z")

</div>

> [@jasongrout](#):
>
> I’m happy to pair with someone for an afternoon (probably not in the next week or so, but before the end of the month) to get them started on writing a JupyterLab extension that would introduce a new notebook type with a custom markdown renderer, which could communicate to the kernel to get user expressions to render.

That’s a wonderful offer @jasongrout, thanks so much! I’d certainly join in just to learn a bit more about the APIs, and if you don’t mind, I think having a recording of this (totally informal, live, as-it-goes) could be very useful for others as well!

---

<div class="post-metadata">

**Author:** ![fcollonval](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/fcollonval/32/801_2.png) [@fcollonval](https://discourse.jupyter.org/u/fcollonval)\
**Post date:** [September 10, 2021, 6:29am UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/64 "2021-09-10T06:29:52Z")

</div>

Hey I would like to help with this. I set up a doodle to start this experiment based on @jasongrout time slots suggestion.

[https://doodle.com/poll/b9gga38v5q6q2pig?utm\_source=poll&utm\_medium=link](https://doodle.com/poll/b9gga38v5q6q2pig?utm_source=poll&utm_medium=link)

---

<div class="post-metadata">

**Author:** ![fcollonval](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/fcollonval/32/801_2.png) [@fcollonval](https://discourse.jupyter.org/u/fcollonval)\
**Post date:** [September 10, 2021, 6:32am UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/65 "2021-09-10T06:32:39Z")

</div>

A technical suggestion, what about using the existing [cell attachments](https://nbformat.readthedocs.io/en/latest/format_description.html#cell-attachments) to store the expression mime bundle.

---

<div class="post-metadata">

**Author:** ![agoose77](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/agoose77/32/3455_2.png) [@agoose77](https://discourse.jupyter.org/u/agoose77)\
**Post date:** [September 10, 2021, 8:06am UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/66 "2021-09-10T08:06:59Z")

</div>

@fcollonval honestly, this seems like a good idea. I considered it yesterday, but I think I was pushing harder to do something more radical. A 👍 from me!

---

<div class="post-metadata">

**Author:** ![agoose77](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/agoose77/32/3455_2.png) [@agoose77](https://discourse.jupyter.org/u/agoose77)\
**Post date:** [September 10, 2021, 2:53pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/67 "2021-09-10T14:53:47Z")

</div>

OK, I’ve created an initial repository to prototype this, and given @fperez, @fcollonval, and @jasongrout access.

So far, this repo only contains a Markdown-it tokenizer implementation. This does work in standalone, but I noticed that I couldn’t see the set `data-expression` attribute in the Chrome inspector. Perhaps a sanitizer thing?

Anyway, the _hard_ part is re-implementing a Markdown cell given that lots of the core logic is private.

---

<div class="post-metadata">

**Author:** ![fperez](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/fperez/32/148_2.png) [@fperez](https://discourse.jupyter.org/u/fperez)\
**Post date:** [September 10, 2021, 3:46pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/68 "2021-09-10T15:46:07Z")

</div>

Thanks so much @agoose77 for moving forward! Exciting to see this idea re-started after so many early attempts 🙂

And I’m +1 on the cell attachments approach too, I was actually pondering that as I realized that the metadata dicts could get pretty unwieldy if potentially larger outputs come in. You don’t expect a small “metadata post it” (which is how I think of them) to all of a sudden have a large base64-encoded image in it!

This will be great to explore and play with. Full disclosure: I’m so overloaded with other things in the project that I can’t realistically expect to contribute too much, but I’m delighted to be along for the ride. I’ll do my best to learn, pitch in when I can, and don’t worry about ever bottlenecking on me. If I’m not around, carry on happily with the rest of the community and I’ll catch up as I can, always grateful for people putting their energy into the project.

---

<div class="post-metadata">

**Author:** ![jasongrout](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/jasongrout/32/371_2.png) [@jasongrout](https://discourse.jupyter.org/u/jasongrout)\
**Post date:** [September 10, 2021, 5:21pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/69 "2021-09-10T17:21:14Z")

</div>

> [@fcollonval](#):
>
> Hey I would like to help with this. I set up a doodle to start this experiment based on @jasongrout time slots suggestion.

I won’t be able to pair for a significant amount of time in the next two weeks, but I could quickly go over my notes from above (i.e., 30-45 minutes) in the next week, and then it probably makes most sense to make sure someone that understands jlab extensions deeply is there. I could pair for a more significant amount of time (like half a day or something, and doesn’t have to be an afternoon) in the last week of September.

---

<div class="post-metadata">

**Author:** ![stefanv](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/stefanv/32/5073_2.png) [@stefanv](https://discourse.jupyter.org/u/stefanv)\
**Post date:** [September 10, 2021, 11:35pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/70 "2021-09-10T23:35:48Z")

</div>

Great, I’d be up for both! 🙏

---

<div class="post-metadata">

**Author:** ![agoose77](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/agoose77/32/3455_2.png) [@agoose77](https://discourse.jupyter.org/u/agoose77)\
**Post date:** [September 13, 2021, 8:55pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/71 "2021-09-13T20:55:09Z")

</div>

@fcollonval and I had a meeting today to talk about this, and we’ve got a proof-of-concept working. This still needs a lot of work, but it’s fun to play around with!

![preview](https://canada1.discourse-cdn.com/flex031/uploads/jupyter/original/2X/8/8991868e061a7a36a13ef06a3be48ae7331e93fe.gif)

@jasongrout when prototyping this, I ran into a few places where the existing API for the notebook was pushing me to shadow large parts of `MarkdownCell` because it was private. Where’s the best place to discuss this part of the codebase e.g. to think about improving the API?

[![](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/agoose77/jupyterlab-imarkdown.git/master?urlpath=lab)

---

<div class="post-metadata">

**Author:** ![stefanv](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/stefanv/32/5073_2.png) [@stefanv](https://discourse.jupyter.org/u/stefanv)\
**Post date:** [September 13, 2021, 10:36pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/72 "2021-09-13T22:36:37Z")

</div>

That is neat @agoose77! In this prototype, do you only replace variables, or can you execute arbitrary inline statements?

---

<div class="post-metadata">

**Author:** ![agoose77](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/agoose77/32/3455_2.png) [@agoose77](https://discourse.jupyter.org/u/agoose77)\
**Post date:** [September 13, 2021, 10:38pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/73 "2021-09-13T22:38:10Z")

</div>

This can evaluate an expression and render the result (subject to some bugs, e.g. `Label` widgets don’t seem to work right now). The formatting is not quite right yet either; raw `text/plain` is formatted as `<pre>` which is a block element.

Notebooks should save the MIME bundles in `cell.attachments`, but currently JupyterLab has some naive logic for wiping unreferenced attachments on save, so rendered outputs aren’t saved properly.

---

<div class="post-metadata">

**Author:** ![choldgraf](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/choldgraf/32/1264_2.png) [@choldgraf](https://discourse.jupyter.org/u/choldgraf)\
**Post date:** [September 13, 2021, 10:46pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/74 "2021-09-13T22:46:31Z")

</div>

This is super cool - what can I do to help either improve this or push this forward? I bet that @rowanc1 would be interested in providing input and ideas as well as I think CurveNote has built similar kinds of functionality into their UI, so maybe they have some UX experience that’d be useful

---

<div class="post-metadata">

**Author:** ![agoose77](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/agoose77/32/3455_2.png) [@agoose77](https://discourse.jupyter.org/u/agoose77)\
**Post date:** [September 13, 2021, 11:05pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/75 "2021-09-13T23:05:09Z")

</div>

@fcollonval is looking into alternative Markdown syntax (which would potentially remove the `jupyterlab-markup` dependency (which is good for core, as I suspect moving JLab to Markdown-It would be a major change)). This would perhaps be a good area to discuss. If we _don’t_ use something that we already parse well (like a link syntax), then we’d need to further modify our Markdown renderer.

In theory, the JLab side of things hasn’t got too much left to do for this PoC. There are a few bugs, but otherwise it works as is. The next piece of the puzzle is figuring out `nbconvert`. Ignoring the `execute` feature, `nbconvert` should already have the MIME bundles in the `MarkdownCell.attachments` metadata, so it’s a question of implementing a renderer.

We also need to discuss how things should render when there _aren’t_ any cell attachments available (e.g. red warning emojis?!).

---

<div class="post-metadata">

**Author:** ![choldgraf](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/choldgraf/32/1264_2.png) [@choldgraf](https://discourse.jupyter.org/u/choldgraf)\
**Post date:** [September 14, 2021, 12:17am UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/76 "2021-09-14T00:17:20Z")

</div>

Sounds good - two quick thoughts:

- Perhaps you’d be interested in this thread, which was inspired by this conversation and which I think could make it easier to support this at the `execute` level in nbconvert [Create some extension points for notebook / cell execution · Issue #158 · jupyter/nbclient · GitHub](https://github.com/jupyter/nbclient/issues/158)
- IMO the syntax choice shouldn’t be constrained _at the design phase_ by implementation questions like “what markdown parser should we use”. That’s fine for prototypes, but ultimately we should try to use the best-possible syntax in general, not restrict it to whatever marked.js already knows how to parse

---

<div class="post-metadata">

**Author:** ![psychemedia](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/psychemedia/32/24_2.png) [@psychemedia](https://discourse.jupyter.org/u/psychemedia)\
**Post date:** [September 14, 2021, 2:52pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/77 "2021-09-14T14:52:10Z")

</div>

Nice demo, but it begs a question…

If you have a code cell:

```auto
a = 1

```

A markdown cell:

```auto
The value of `a` is {{ a }}

```

And code cell:

```auto
a = 2
print(f"The printed display of a is {{ a }}")
display(f"The displayed display of a is {{ a }}")

```

then you run all three cells, what does the markdown cell do?

If you now change the value of `a` in the first cell and run it, does the markdown cell update? What what about the other references to `a`? Are just the markdown cells reactive? Is cell order in the notebook relevant? (Channel your inner Joel Grus to consider the “hidden state” fallout etc! 😉

---

<div class="post-metadata">

**Author:** ![agoose77](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/agoose77/32/3455_2.png) [@agoose77](https://discourse.jupyter.org/u/agoose77)\
**Post date:** [September 14, 2021, 2:54pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/78 "2021-09-14T14:54:19Z")

</div>

These are indeed “things to be wary of”, but they also apply to the rest of the notebook. In Jupyter Notebooks (ignoring [akernel](https://github.com/davidbrochart/akernel)), cell ordering is important, and side effects are entirely “supported”!

---

<div class="post-metadata">

**Author:** ![psychemedia](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/psychemedia/32/24_2.png) [@psychemedia](https://discourse.jupyter.org/u/psychemedia)\
**Post date:** [September 14, 2021, 3:00pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/79 "2021-09-14T15:00:04Z")

</div>

🙂

I also wonder about a markdown cell such as:

_What happens to {{ a }} if we add 1 as per {{ a + 1}} or if we try to make an assignment {{ a=1; a}}_

What happens if `a` is a mutable list or a dict? Can it be changed from the markdown cell?

I think it makes most sense if markdown can only literally display a value in terms of maintaining the integrity and transparency of the code. That said, being able to influence the format of the output value (for example formatting a float to 1dp) would be useful.

(I’ve also been pondering `akernel` in the context of this thread…)

---

<div class="post-metadata">

**Author:** ![agoose77](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/agoose77/32/3455_2.png) [@agoose77](https://discourse.jupyter.org/u/agoose77)\
**Post date:** [September 14, 2021, 3:06pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/80 "2021-09-14T15:06:37Z")

</div>

Only “expressions” can be evaluated in this context (which depends upon the interpretation of “expression” by the kernel in use). For IPython, this means statements are not legal, nor are the walrus assignments (which is an expression, but forbidden at root level).

If we make a stateful callable, e.g.  
 ![image](https://canada1.discourse-cdn.com/flex031/uploads/jupyter/original/2X/3/35279b54a579d56b271f9ed7b20f281d609ffc41.png)  
then we can easily introduce non-reproducible outputs. But that holds for code cells, as before 🙂

If we want to start adding rules to this, then we deviate from “expression” as determined by the kernel, which becomes a lot more problematic as the frontend has to start understanding the language of the kernel.

---

<div class="post-metadata">

**Author:** ![matthew.brett](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/matthew.brett/32/2204_2.png) [@matthew.brett](https://discourse.jupyter.org/u/matthew.brett)\
**Post date:** [September 14, 2021, 3:13pm UTC](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/81 "2021-09-14T15:13:16Z")

</div>

My model of the inline Markdown is that the inline Markdown snippets get ‘executed’ when it the cell is rendered, just as the code cell does. The inline system could allow:

- Variables only or
- Expressions or
- Abitrary code.

So, your question boils down to whether one of your inline snippets raises an error or not.

Someone reasonably asked what to do about stdout, and about cell execution numbers - and I guess we could avoid those two questions by allowing variables only - but I suspect that there would quickly be a use for more complex expressions. Maybe we could mine some R notebooks to see what people have got up to there?

[Previous page](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525.md?page=3)

[Next page](https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525.md?page=5)