Running Jupyter Notebook with Different Configurations from CLI

Notebook
1

Hey everybody!

I started to use Jupyter Notebook for my project and I use nbconvert to run it from the command line.
I want to know that is there any way to have a .ipynb file and run it with some command-line arguments or reading from stdin somewhere?
I read nbconvert documentation but I didn’t find out a way to do it.

If there is no way to do something like those, I want to know what is the best practice to have a single .ipynb file and run with different configurations?

Thank you in advance.

2

I guess that if you want a script that uses the command line interface, using a Python script instead of a Jupyter Notebook is much more common. Now I don’t know why you prefer Jupyter Notebook over a plain script, maybe you need to elaborate on that. Maybe Jupyter is just an overkill once you are done with development and it was only good for debugging during development?

If you insist that it must stay a Jupyter Notebook: One idea I had was maybe you can create a regular Python script that serves the CLI (there are great modules like argparse that help you here) and import your Jupyter Notebook as a module as described here?

1 Like
3

Thanks for your reply.
Actually most of my project is based on having an explanation over results and peace of code, therefore, I chose Jupyter Notebook to satisfy this requirement.
But also I don’t want to have duplicate code for some common functionalities, so I tried to pass command-line arguments or reading from stdin to avoid it.
According to your reply, I understand that there is no way to just use Jupyter Notebook and nbconvert to satisfy my second requirement, right?
And also thanks for your idea about importing Jupyter Notebook in a Python script. Maybe I use plain Python scripts for specific functionalities to avoid duplication.

4

You can have a look at papermill to parameterise a jupyter notebook (add different configurations/values to a variable), check out https://papermill.readthedocs.io/en/latest/usage-parameterize.html

We at econ-ark, created a script which loads in all the configurations using a YAML file (https://github.com/econ-ark/PortfolioChoiceBlogPost/blob/d68c227a4ec2b5116b7e6500efd5342d9e1a27bd/config.yaml) and loading that in the jupyter notebook, followed by nbconvert.

1 Like
5

Nice!
It is exactly what I was looking for.
Thanks a lot :star_struck: