I’m executing a python script code.py from my jupyter notebook by executing !python code.py . My notebook is not showing the print statements that are called during execution. Instead its printing everything together after the end of execution. Please help! (Same isssue in jupyter lab as well)
First, you probably would be better off not using the ! command to run python from scripts inside a Jupyter notebook, in general. Jupyter (and IPython) have the %run magic command for running Python script files which offers better integration and many more options, see here for much more detail on the further options it provides.
Specifically, I suspect you may want to try with %run -i code.py in your notebook cell and see if that gets closer to the behavior you’d like.
That ! calls out to a separate unix shell process from your notebook and executes that and all those layers can add delay if your tasks are computationally demanding. (This will make more sense after ‘buffering’ is covered below.) Since it isn’t a bash shell command you are running, you’ll have more options by using %run code.py inside your notebook. This is actually an IPython magic that Jupyter inherits.
Even using %run you can see delay and what seems to be sub-optimal return of the output as the task runs. I’ve see it most often when mixing use of sys.stderr.write() and print() during development. This is because output to stdout which print() would access is buffered while stderr is unbuffered. This post explains it pretty succinctly. Most times it doesn’t really matter, but it may in your case and hopefully some suggestions on that page may help you.
However, if that didn’t help you much, you may want to better specify what you are trying to achieve. As there are other related options such as import code and %load. There’s even a possibility what you are attempting to achieve would need to invoke pauses with time.sleep() or work better with the use multiprocessing, multithreading, or even asyncio if you need absolute control over when things happen, and it is all being done with pure Python.
1 Like