Should Jupyter recommend a text-based representation of the notebook?

This is great discussion! I’ve started similar thread Jupyter and GitHub - alternative file formant , I’m glad it’s already being thought of.

I was experimenting with similar approach to jupyter-format - namely content_manager class as means to allow users to work on text-format notebooks transparently.

I was experimenting with markdown + yaml mix that would look similar to this:

MyST looks very promising! Is there any work to create MyST notebook content manager?

1 Like

Not yet, but that’s mostly just because we’re running on limited resources and time right now. Our plan for MyST is to see how people use it, what they like about it, whether it gains more adoption and interest from the community. We’d also like to add extensions (e.g. a Jupyter Lab extension, or a notebook extension) that add functionality for MyST markdown in Jupyter interfaces (this would be much easier if https://github.com/jupyterlab/jupyterlab/issues/272 happens, this is because MyST uses a python implementation of markdown-it in the hopes that we can easily port over a MyST parser to Javascript…in fact, some of this is already done as there is now a MyST markdown plugin for vscode).

If at some point there seems to be enough momentum and we think MyST has made the right technical and syntax choices, then we can build in more “core” support and, one day, potentially try and standardize some of its syntax within Jupyter. But I think it’d be a long road to that point - as I mentioned in the other thread, it would need to be a community discussion and decision, probably a JEP.

(happy to chat more about myst if it’s helpful)

I think the way pandoc exports ipynb files is really interesting. Given the ubiquitous use of pandoc and the ability to convert to other formats (notably latex), IMHO it is a strong candidate.

@choldgraf what were the reasons to develop a new format?

Some of our reasoning is detailed here: https://myst-parser.readthedocs.io/en/latest/#why-myst-markdown and the notebook version here: https://myst-nb.readthedocs.io/en/latest/use/markdown.html

the tl;dr is: we really like markdown, we wanted something that could handle more complex use-cases than CommonMark. We really like Sphinx and think it has most of the features one would want for publishing. We also didn’t want to force people to use reStructuredText, and so we created MyST, which is an attempt at “best of both worlds” for reStructuredText and markdown. (side note: we also wanted something that was heavily inspired by RMarkdown)

It seems that the MyST and pandoc markdown formats are really close. The major difference is the directives.

E.g. codebraid uses pandoc’s markdown. One can further break blocks with :::.

One idea that I am exploring is to build the document interactively with jupyter, export it to pandoc’s markdown, execute with codebraid, and convert to latex.

I understand that jupyter book is aiming for a similar goal using sphinx.

Indeed - and we’re also discussing adding ::: syntax here: https://github.com/executablebooks/MyST-Parser/issues/154

For reference, below is the output from pandoc (there is no \ in front of the python block, added them to be able to display).

PS1: Used the command line options --atx-headers and --standalone
PS2: pandoc supports metadata and tags as well.

---
jupyter:
  kernelspec:
    display_name: Python 3
    language: python
    name: python3
  language_info:
    codemirror_mode:
      name: ipython
      version: 3
    file_extension: .py
    mimetype: 'text/x-python'
    name: python
    nbconvert_exporter: python
    pygments_lexer: ipython3
    version: 3.8.2
  nbformat: 4
  nbformat_minor: 4
---

::: {.cell .markdown}
# Section

This is a section.

Images are inserted with the following syntax:

![image](markdown.png)

And equations as $y=a+bx$.
:::

::: {.cell .code execution_count="1"}
\``` {.python}
x = 2
y = x*x
print(y)
\```

::: {.output .stream .stdout}
    4
:::
:::