`jupyter run` requires notebook to be previously run

Hi. Since nbclient v0.5.9 jupyter has had jupyter run (c.f. Command-line interface: jupyter execute by palewire · Pull Request #165 · jupyter/nbclient · GitHub and Changed the default command to `jupyter run`, but left in `execute` as an alias with a deprecation warning by palewire · Pull Request #173 · jupyter/nbclient · GitHub), which is great. However, it seems that jupyter run only works if the notebook that it is operating on has already been run (that is has execution counts that are non null)

As an example: A 1 cell Jupyter notebook named debug.ipynb with print("hello world").

Executing this at the command line works as expected.

(jupyter-run-debug) feickert@ThinkPad-X1:~/Code/debug/jupyter-run-debug$ jupyter run debug.ipynb
{'cells': [{'cell_type': 'code',
   'execution_count': 1,
   'id': 'aec173a3',
   'metadata': {},
   'outputs': [{'name': 'stdout',
     'output_type': 'stream',
     'text': ['hello world\n']}],
   'source': ['print("hello world")']}],
 'metadata': {'kernelspec': {'display_name': 'Python 3 (ipykernel)',
   'language': 'python',
   'name': 'python3'},
  'language_info': {'codemirror_mode': {'name': 'ipython', 'version': 3},
   'file_extension': '.py',
   'mimetype': 'text/x-python',
   'name': 'python',
   'nbconvert_exporter': 'python',
   'pygments_lexer': 'ipython3',
   'version': '3.9.6'}},
 'nbformat': 4,
 'nbformat_minor': 5}(jupyter-run-debug) feickert@ThinkPad-X1:~/Code/debug/jupyter-run-debug$ [IPKernelApp] WARNING | Parent appears to have exited, shutting down.

However, if you “Restart & Clear Output” and save (removing the execution count for the cells from the notebook)

and run again you get an error

(jupyter-run-debug) feickert@ThinkPad-X1:~/Code/debug/jupyter-run-debug$ jupyter run debug.ipynb
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
/tmp/ipykernel_31410/1794071965.py in <module>
      3   {
      4    "cell_type": "code",
----> 5    "execution_count": null,
      6    "id": "aec173a3",
      7    "metadata": {},

NameError: name 'null' is not defined
Traceback (most recent call last):
  File "/home/feickert/.pyenv/versions/jupyter-run-debug/bin/jupyter-run", line 8, in <module>
    sys.exit(RunApp.launch_instance())
  File "/home/feickert/.pyenv/versions/3.9.6/envs/jupyter-run-debug/lib/python3.9/site-packages/jupyter_core/application.py", line 264, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/home/feickert/.pyenv/versions/3.9.6/envs/jupyter-run-debug/lib/python3.9/site-packages/traitlets/config/application.py", line 846, in launch_instance
    app.start()
  File "/home/feickert/.pyenv/versions/3.9.6/envs/jupyter-run-debug/lib/python3.9/site-packages/jupyter_client/runapp.py", line 110, in start
    raise Exception("jupyter-run error running '%s'" % filename)
Exception: jupyter-run error running 'debug.ipynb'
(jupyter-run-debug) feickert@ThinkPad-X1:~/Code/debug/jupyter-run-debug$ [IPKernelApp] WARNING | Parent appears to have exited, shutting down.

The NameError of course “makes sense”, but it seems like Jupyter should be able to handle this so that you can execute notebooks from the command line if they have had all output stripped for versioning reasons (let’s ignore the existence of Jupytext for the time being).

Is this a bug? Or am I missing a CLI option to avoid this?

For completeness, here’s my Python 3.9.6 virtual environment arrived at by

$ python -m pip install --upgrade pip setuptools wheel
$ python -m pip install jupyter
$ python -m pip freeze > requirements.txt
$ cat requirements.txt
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
attrs==21.2.0
backcall==0.2.0
bleach==4.1.0
cffi==1.15.0
debugpy==1.5.1
decorator==5.1.0
defusedxml==0.7.1
entrypoints==0.3
ipykernel==6.6.0
ipython==7.30.1
ipython-genutils==0.2.0
ipywidgets==7.6.5
jedi==0.18.1
Jinja2==3.0.3
jsonschema==4.3.0
jupyter==1.0.0
jupyter-client==7.1.0
jupyter-console==6.4.0
jupyter-core==4.9.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.2
MarkupSafe==2.0.1
matplotlib-inline==0.1.3
mistune==0.8.4
nbclient==0.5.9
nbconvert==6.3.0
nbformat==5.1.3
nest-asyncio==1.5.4
notebook==6.4.6
packaging==21.3
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
prometheus-client==0.12.0
prompt-toolkit==3.0.24
ptyprocess==0.7.0
pycparser==2.21
Pygments==2.10.0
pyparsing==3.0.6
pyrsistent==0.18.0
python-dateutil==2.8.2
pyzmq==22.3.0
qtconsole==5.2.2
QtPy==1.11.3
Send2Trash==1.8.0
six==1.16.0
terminado==0.12.1
testpath==0.5.0
tornado==6.1
traitlets==5.1.1
wcwidth==0.2.5
webencodings==0.5.1
widgetsnbextension==3.5.2

jupyter run is actually a jupyter-client CLI, which runs a Python script, not a notebook.
nbclient has a jupyter execute CLI, which is what you should use. We renamed it to jupyter run at some point but realized it conflicted with jupyter-client, so we reverted the change. Sorry for the confusion.

1 Like

Thanks! Indeed, I saw Changed the default command to `jupyter run`, but left in `execute` as an alias with a deprecation warning by palewire · Pull Request #173 · jupyter/nbclient · GitHub and didn’t look any further for followup or reverts. But jupyter execute is working as expected now, which is fantastic! :slight_smile: :+1:

(jupyter-run-debug) feickert@ThinkPad-X1:~/Code/debug/jupyter-run-debug$ jupyter execute debug.ipynb 
[NbClientApp] Executing debug.ipynb
[NbClientApp] Executing notebook with kernel: python3
1 Like