One of the issues with using notebooks in a code development workflow (and yes, I know… JupyterLab… but I prefer notebooks for several reasons…).
A common example is the mess you can get in with half-working notebooks used as scratchpads, and code management around code within and derived from the notebooks.
One of my attempts at managing this is to load notebooks as modules into other notebooks. The notebook modules are littered with
if __name__=='__main__': guards in some of the code cells; these guards allow me to call on code fragments for testing and previewing the outputs of functions defined within that module notebook when I run / check/ develop code explicitly within the module notebook.
I’ve also started using Jupytext to allow me to write text and code in markdown (
.md) files edited and run as notebooks (the code cell outputs aren’t saved to the md file, just the md and code source; a “dual” notebook, with outputs, can also be automatically saved alongside the md file). Checking the markdown file into git makes Github diffs useful again.
Equally, I could use Jupytext to let me author notebooks saved as Python (
Now I’m wondering what I’d need to do to let me write code in Python files (
.py) rendered as editable notebooks using Jupytext, without lots of
if __name__=='__main__': baggage.
I need to check that Jupytext respects RawNBConvert cells (about) but if it does, I wonder if the following workflow would make sense:
- edit the py file as a notebook under Jupytext, with code from previously guarded code cells in the RawNBConvert cells marked as ‘Python’ RawNBConvert cells;
- the py file can be loaded as a module into other py files or notebooks in the normal way (none of the special machinery required to load notebooks into notebooks, etc);
- (the voodoo magic step) when viewing a .py file as a notebook under Jupytext, if an NBConvert format cell is set to ‘Python’, treat it as an executable code cell.
Now I should be able to have a messy .py file with lots of code I can execute when I view it as a notebook under Jupytext, but that isn’t executed if I load the
.py file in as a module anywhere?
Or is that all just crazy talk?