Jupyter and GitHub - alternative file format

Hello everyone, thanks for discussing this!

@inc0, with Jupytext 1.5 you can easily create text-only notebooks:

As you see there are many different formats. I think this is because people sometime need more than a Jupyter Notebook. For instance, Myst-MD notebooks work well with Sphinx and allow cross references and citations, a feature that does not exist in plain Jupyter notebooks. Notebooks represented as scripts work well in IDEs (for instance, the percent format is understood by Spyder, Hydrogen, VS Code and PyCharm). In your case, I think you are looking for a format that fits nicely on GitHub.

Currently Jupytext’s text-only notebooks don’t store outputs. We have an issue to discuss how we could store the outputs in Markdown notebooks. I think outputs should not be in the main text document but in separate files, in a directory named after the notebook (not unlike for the SageMath described by @jasongrout). For instance, if my output is a pandas data frame, I want it to be stored in unnamed_cell_55.html (or even better world_pop.html if the user gave a name to the code cell) and then include it in my main notebook. But not in the notebook itself, otherwise changing one line of code would create hundred lines of diff on the main file.

Concretely, one reason for which we have not finalized outputs in Markdown is that, according to my experiments, it is not possible to display an embedded HTML file in a Markdown file on GitHub. But maybe it is? If not, do you think this is something that could change?

PS: I like the hidden YAML div for the metadata in your Markdown sample notebook. The proposal also appeared at https://github.com/mwouts/jupytext/issues/527, and it’s scheduled for the next version of Jupytext.

3 Likes