During the Pangeo meeting today with @rabernat , @jhamman , @jsignell, @jwagemann, @robfatland, @amanda-tan, and @mrocklin, the question “How do you keep your notebooks fresh?” came up and there currently isn’t an agreed upon solution for testing notebooks and detecting when they need to be updated. T…

This suggestion hits an important scenario. Generally speaking, we would like to have an un-executed version of the notebook to run in binder. That’s what most of our existing example binders do. We don’t want the cells to be run, because we want the users to have the experience of seeing the output…

you could totally do that with papermill!

If we did this, how could we use CI to view the output (rendered notebooks) of a PR to a repo? I can’t quite imagine how that would work.

Take a look at fastai and quantecon’s tooling. IIRC both groups are autotesting notebooks for their courses and websites.

This is a fairly long post linking out to various building blocks to illustrate a solution I once built for a customer. The whole system uses CircleCI to execute several (unexecuted) notebooks with papermill, turns them into HTML, stores that HTML as artefact, and has a bot that will post a link to…

@willirath Have you by any chance posted a write up of how you went about setting up the nbval notebook testing? I need to do something similar for some course notebooks and keep putting it off because I’m not sure where to start with things like travis, circle.ci etc. thanks –tony

Am not allowed to put more than two links in a post… So the refs are a bit weird below. Parcels repo: https://github.com/OceanParcels/parcels/ Requirements See environment_py3_linux.yml in the Parcels repo for full env. They get pytest and nbval from PyPi, and py (why though?) from conda-forge. …

[image] willirath: Am not allowed to put more than two links in a post… You can head to Introduce yourself! - #59 by josephcslater to introduce yourself to gain more karma. I’ll see if I can fix up some of the links for you.

This second post seems to have lifted me above the initial barrier.

Testing notebooks

Binder

willirath April 4, 2019, 7:34am 10

Parcels uses nbval to ensure all tutorials work.

Topic		Replies	Views
Binder Notebook Builder Bot Binder	6	1317	April 14, 2020
Binder as part of a test framework Binder	5	765	December 3, 2018
Creating a future infrastructure for notebooks to be submitted and peer-reviewed Publishing	25	5190	September 17, 2020
Embed binder-related metadata in notebook? Binder	8	1420	August 11, 2021
Executing .ipynb on github? General	20	1310	September 10, 2025

Testing notebooks

Related topics