Selective Execution of Jupyter Notebook Cells

Notebook
1

I utilize Jupyter Notebook for my machine learning projects, appreciating its capacity to integrate code and documentation. However, I’ve encountered a challenge and have tried to identify potential solutions.

My Jupyter Notebook Project encompasses the subsequent tasks:

  1. Pre-training a neural network
  2. Finetuning the neural network

My objective is to enable the execution of either of the following:

  1. Solely pre-training the network
  2. Pre-training the network followed by finetuning

I’ve discovered that Jupyter Notebook offers the option to apply tags, and it seems like an idea to implement a mechanism where I can designate two parameters: pretraining and finetuning, as demonstrated below:

  • If pretraining is set to True, activate the “pretraining” tag.
  • If finetuning is set to True, activate the “finetuning” tag.

Is there a feasible approach to achieve this functionality? (An example below with the tags)

image
image816×374 10.9 KB

2

Table of contents in JupyterLab 3 offers running cells by tag. This was not yet implemented in JuypterLab 4/Notebook 7 I think. GitHub - jtpio/ipylab: Control JupyterLab from Python Notebooks with Jupyter Widgets 🧪 ☢️ 🐍 is one way to control execution programatically.

1 Like
3

If you’re ok doing this from the terminal, you can use ploomber-engine. ploomber-engine allows you to run notebooks from the terminal and it has an option to remove cells with certain tags before execution, so you can do:

ploomber-engine input-notebook.ipynb pre-training.ipynb --remove-tagged-cells fine-tuning

ploomber-engine input-notebook.ipynb fine-tuning.ipynb --remove-tagged-cells pre-training