`command not found` message in Notebook for installed package

JupyterLab
1

I have JupyterLab Version 4.3.6 installed in a Python venv. I have Python package datamodel-code-generator installed within the venv. If I run a command like datamodel-codegen --input DatasetSchema.json --input-file-type jsonschema --output gen.DatasetModel.py from the venv then the command processes as expected. However if I open a Notebook within JupyterLab running on this same venv, when I run a Python code block with !datamodel-codegen --input DatasetSchema.json --input-file-type jsonschema --output gen.DatasetModel.py the output message is /bin/bash: line 1: datamodel-codegen: command not found. The command exists in the venv so I’m wondering why the Notebook is unable to see it? When I run an install script for datamodel-code-generator within the Notebook, the Notebook says it is already installed. So the Notebook is smart enough to know the package is installed but it still does not see the datamodel-codegen command. Maybe this is a bug in the latest JupyterLab version? I was able to get this code to work in another JupyterLab environment running in a Docker container.

2

This is a common issue because the command line in your venv and the shell process that !datamodel-codegen gets sent off to are not necessarily the same thing. In your case, here you have experiential proof they aren’t the same. It happens outside of a venv, too, as often users are confused that installed doesn’t mean installed to same environment your kernel is using.

If this is the only package you are having an issue with you have some options for making it work:

a) The easiest is to install it from inside the running notebook using the appropriate magic install commands:

%pip install datamodel-code-generator

-or-

%conda install conda-forge::datamodel-code-generator

The latter being if you used Anaconda/conda so that conda is now your primary package manager.
The % symbol is important here. Using the magic commands will ensure it installs to the same enivronment your kernel is using. See here for more about the modern magic commands that can be helpful to get past hurdles such as these by ensuring installation to the correct environment without you needing to sort it all out by hand. Please restart the kernel after that and since this installs to a command you may even want to start everything over again.

b) If it works to run the command like datamodel-codegen --input DatasetSchema.json --input-file-type jsonschema --output gen.DatasetModel.py from the venv, then try running in that same place:

which datamodel-codegen

Then using that absolute path, you may be able to call it from within the active Juptyer notebook, like so:

!/srv/conda/envs/notebook/bin/datamodel-codegen ...

Which is the equivalent command that worked in my test case. (Yours will probably be much different but the idea is the same.) However, no guarantees that once you get the absolute path that it will work because the caveat is !datamodel-codegen ... worked in my case already, and so maybe my ‘test case’ isn’t exactly like your situation.

c) Usually there isn’t a command line command involved, but if you want to sort it out more directly, see here and errorLogger’s comment here. Plus, here and here will give you some additional insight into what is happening and options for sorting it out. I don’t think those don’t involve a command that gets installed to the command line though so pay special attention to anything that looks helpful for that.

2 Likes
3

Thank you @fomightez for setting me on the right path. It turns out when I ran

! which pip

…I was not running within the venv as I expected. Oh, I was running the jupyter-lab package that was installed on the venv, but that is a completely different thing than running the venv. I did not have the venv activated as I expected. Running:

! which pip`

…from the Notebook showed me my problem. Once I activated the venv the Notebook code behaved as I expected.

As a side note, I use a Makefile to run JupyterLab so I had to change my commands from:

runJLab:
       $(VENV_DIR)/bin/jupyter-lab

… to

runJLab:
	( \
       . .venv/bin/activate; \
       $(VENV_DIR)/bin/jupyter-lab; \
    )

Again, thank you @fomightez for setting me straight. :slight_smile:

1 Like