Python in terminal finds module, jupyter notebook does not

Notebook
1

Hi, I am in a situation where python, in a terminal, finds a module with “import”. In fact, with
“print(help(‘modules’)”
I see a list of all available modules, and the module in question is listed.

However, when running a jupyter notebook, of the proper version of python, I always get “module not found”.

One think that I would like to do is to find out exactly where the module in question is being stored and then append that path to the path where the notebook is looking. But, I don’t know how to find the module path.

Is there another way to go about this?

Thank You
Tom

2

Does the terminal you get from the Jupyter Dashboard allow import? Or are you talking about your terminal software on your computer?

3

Thank you for the reply. Yes, the terminal that I am referring to is from the jupyter dashboard. I would have expected that the terminal and the notebook would be looking in the same place, which, I believe they do.

Here is the result of “sys.path” in the terminal
[’’, ‘/usr/lib/python36.zip’, ‘/usr/lib/python3.6’, ‘/usr/lib/python3.6/lib-dynload’, ‘/usr/local/lib/python3.6/dist-packages’, ‘/u
sr/lib/python3/dist-packages’]

Here is the result of sys.path in the jupyter notebook.
[’’,
‘/usr/lib/python36.zip’,
‘/usr/lib/python3.6’,
‘/usr/lib/python3.6/lib-dynload’,
‘/usr/local/lib/python3.6/dist-packages’,
‘/usr/lib/python3/dist-packages’,
‘/usr/lib/python3/dist-packages/IPython/extensions’,
‘/root/.ipython’,
‘~/jupyter_notebooks/BNN-PYNQ’,
‘~/jupyter_notebooks/BNN-PYNQ/bnn’]
(I added the last two entries manually).

If I had to guess I would think that it was an issue between the kernel that the notebook was executing and the version of python that the terminal was using. I do have to specifically invoke python3.6 in the terminal in order for python to find the module. Python2.7 is the default, which does not find the module.

The result of “print(python_version())” in the notebook is 3.6.5. So I know that it is not running 2.7

Tom

4

If you dont find module, it means that terminal and jupyter notebook are in different environments.
I saw the sys.path:

‘/usr/local/lib/python3.6/dist-packages’,
‘/usr/lib/python3/dist-packages’,

It’s just my guess. Could you show more info about your envrionment.

5

No clue here really, but does this also happen if you open your terminal from different directories? If the module is available in directly in the current dir, that might make it succeed.

6

Yes, I think @00Kai0 is on the right track. However, I don’t know what to tell you to fix it at this point.
When I was switching locally between 2.7 and Python 3, I ran Python 3 and JupyterLab via virtualenv in the hopes of avoiding things like this.
In case you can opt to not be confined to your local machine, this is one of the many reasons I prefer to work remotely with either MyBinder.org or cloud computers or PAAS. I’d be happy to share more information about those with you if that is an option or if no one can save you easily, and you don’t want to move to using a virtualenv locally.

7

00Kai0,
Thank you for your reply.

My environment? Well, I am sure that you mean something very specific. I would be happy to provide you information of a specific nature.

In general, I am running ubuntu 16.04, but the jupyter notebook is running on a xilinx based development board. The Ultra96V2. I can open up terminals on the development board.

According to the documentation for this project, which I got here https://github.com/Xilinx/BNN-PYNQ/

this project was supposed to run once I logged in via a browser after establishing a wireless link, which I have done.

Yes, I can see a number of jupyter notebooks. When I attempt to run one from the browser, I have issues.

What I did was to open up a terminal and convert the ipynb file to a py file. By running the py file I am now able to get most of the project to work. So, I can establish that the hardware on the development system is functional to this point. I got it to work by moving the file in question to a higher level in the directory tree, than was mentioned. This, to me, is a clue but I can not determine what the clue means. By moving the .py file to a higher directory I was able to determine that it could “find” the module in question that it was “barfing” on before (that is a technical term).

However, I did the same thing for the .ipynb file (moving it up to a higher directory) and yes, it found the module in question but the python kernel “hangs” at a certain point, whereas it did not in the .py execution of the project.

So, I am sure that changing the directory is only a partial solution. Where to go from here I am not sure.

Please ask for any specific information that you want.

Thank You

Tom

However, the

2 Likes
8

Try to install the module trough your jupyter instead of your command prompt by this code

**!**pip install module name

whenever you want to tell jupyter that you want to run system command you use ! before your command.

9

@Khosravigashtasb’s advice is outdated in regards to installations.
Inside a Jupyter notebook, use the following in a cell.:

%pip install <module_name>

This installation is now a special Jupyter magic command meant to handle installing to exactly running the current notebook. A similar version of this magic command now works with conda too. Use %conda install as the start of a conda install command.

For more about this see here and discussion here.

@Khosravigashtasb is correct that typically an exclamation point before a general Unix or command line command is indeed a standard Jupyter method. However, in this case it is not a best practice as a better alternative now exists. Additionally, several common Unix-style commands such as ls and pwd don’t require the exclamation point.

1 Like
10

Looks like the OP is running on some embedded hardware, so I don’t think the normal pip directives will be possible for them.

I think that is missing the mark anyway, since they already hinted that simply moving the file makes the import work. I think they actually solved it (and the “hanging” is a completely separate issue—if you don’t get a ModuleNotFoundError, then the import problem is fixed).

It’s hard to recommend a fix since the details of the environment are fuzzy, but it sounds like this problem and it’s not clear that there is a solution other than moving the file or manually configuring the PYTHONPATH:

1 Like
11

Thanks that magic command worked like a charm!

1 Like
12

I have the same problem. I created a virtualenv, activated it, then pip installed 2 libs PyPdf2 and Jp nb, I launched the ipynb from the virtualenv and the results from sys.executable is the global location and not the virtualenv location. My ipynb files are in the directory under data, but even creating a new notebook in the parent directory or virtualenv’s bin directory, the global install is the result of sys.executable, so no packages in site packages are referenced. Appending paths seem to work but when the kernel is restarted you would have to re-append. Seems like you have to tell jp nb via some sort of config file to use the virtualenv vs global, so I think the json path could be correct, maybe launching the jp nb with a command line argument might work, while in a virtualenv instantiation. Also same results as mentioned by OP, the import of PyPDF2 works in the terminal outside Jp nb, inside the terminal of Jp nb, but not in the note book. If I append the virtual env lib/site-package to path or point sys.executable to the virtualenv bin/pyXX, import works, how ever every time the kernel is started you will have to re run those.