Debugger warning: It seems that frozen modules are being used (Python 3.11.0)

JupyterLab
1

This is in a new, clean build using Ubuntu 22.04 LTS and Python 3.11.0 built from source.

A new message is appearing when JupyterLab starts:

0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.

I don’t see any issues opened about this yet. Is there any guidance as to how to proceed? It is clear how to disable the warning via environment variable, but is that the correct choice here? If that isn’t the correct choice, how do I pass -Xfrozen_modules=off to Python? Lastly, is this caused by a problem elsewhere and both choices are in reality not correct?

2

I’d wait to make any big decisions… 3.11 wheels for all platforms will be… a while, though.

A near term will be reproducible binder environments working with 3.11, and the quickest path to a deeper look at this will likely be via the conda-forge python311 migration… looks like pycurl is the only thing missing in the base setup, which is semi-incredible.

It’s also worth looking at whether we slim the core binder deps down even more… though i feel kinda bad doing “exploratory” stuff right now, as every job counts.

1 Like
3

As of an hour ago or so, binders with an environment.yml specifyng python >=3.11 now seem to work.

If you can work up a concrete gist that busts the debugger with frozen modules, we can get down to business.

1 Like
4

I’m just getting started with Jupyter (jupyter-lab) and am getting this warning on a freshly installed virtualenv with python3.11 in Dec 2022. Is there any new info about fixing or suppressing this error?

I figured out that starting with

python -Xfrozen_modules=off -m jupyterlab_server

instead of just

jupyter-lab

avoids the error, but I don’t know whether this is correct or even advisable.

1 Like
5

Any news on this?
I still get the message described above when I try to register a virtual environment in jupyer lab with:
python -m ipykernel install --user --name=vpanel

I use python 3.11.2

6

As before: has this warning actually done anything to impact your experience yet?

7

Well, it has forced me to move back to python 3.10.10…

8

Does the kernel not start? Did you try setting the environment variable?

9

Hello!

I am having the warnings as well in a similar setup. No impact on the experience
except that it tends to pollute logs. For example, when running Jupyter-Book to build
an html book from a collection of notebooks, the warning appears once per notebook
when it is executed. It makes it harder to spot real issues.

Setting PYDEVD_DISABLE_FILE_VALIDATION=1 clears the warning, so there is
an easy workaround.

Thanks for your care!

1 Like
10

I think the issue is that this warning creates uncertainty about how to proceed without a clear explanation of the warning’s context.

Here’s a few questions I imagine could come up when reading this warning:

  • What are frozen modules?
  • What causes the problem with frozen modules and debugger breakpoints?
  • Is there an alternative to frozen modules? If so, what is it, what are the tradeoffs, and how do I change to that alternative if I want to?
  • If I need to use frozen modules, in what situations would the debugger miss breakpoints?
  • Is it safe to turn off frozen modules? If so, in what context?
  • Is it safe to disable this validation? If so, in what context?

Appending one or more documentation links to this warning with further context might be helpful.

2 Likes
11

Appending one or more documentation links to this warning with further context might be helpful.

This is a fine suggestion but may be easier to act on if filled as an issue against GitHub - microsoft/debugpy: An implementation of the Debug Adapter Protocol for Python which is where the code generating the warning resides:

Maybe the solution should be for ipykernel to automatically append -Xfrozen_modules=off flag?

12

The issue linked over the message does answer some of these questions, or more precisely the Python author answers them in this and that and other comments.

1 Like
13

Thanks
But I think that all we need to do is how to pass -Xfrozen_modules=off to python in jupyter.

14

This warning isn’t what bothers me. The inability to debug python code in jupyter is. This warning seems to be the solution to the problem of not being able to debug code.

15

I get the same warning message for Python 3.10.12.

17

I realize this warning has been a bit discussed to death and the problem for interactive jupyter use is really in debugpy.

However, I do still have a question: why does jupyter seem to unconditionally import this package? In particular, does it have to actually be imported when running something like nbconvert / is there a way on the jupyter/ipython side to prevent it beyond uninstalling the package? The case where this error message is the most annoying for me is not interactive use, but in automated testing using nbconvert on many notebooks in sequence.

(From reading ipkernel code it appears a bit worse than just importing: it looks like the debugger is unconditionally set up in full if it is available at all on any ipykernel start – though I could easily be misreading this code, which is not trivial.)

1 Like
18

Yes ipykernel uses imports to check if debugpy is available; on some level this cannot be definitely resolved without import, because the import can error out even if debugpy is installed see `DebuggerInitializationError` kills the kernel · Issue #986 · ipython/ipykernel · GitHub.

This forum (although this thread has many good ideas) is probably not the best place to discuss improvements to ipykernel handling of debugpy (because most people who actually maintain it are not very active here); instead I would recomment opening issues and pull requests on GitHub - ipython/ipykernel: IPython Kernel for Jupyter.

1 Like
19

Yes it loads an empty kernel, I can’t do anything on jupyter.

1 Like
20

Hello,
I am new to Jupyter labs so please excuse my unknowing. I have tried to up date my SSL with pip install certifi. I have tried to stop X-frozen_modues with pluging straight in to the terminal(X_forzen_moduals=off), with each change in terminal, I have restarted the terminal in the virtual emviromment. I have been using Jupyter labs for about a year and just recently has this issue come up.
I use python 3.11 and my version of Jupyter lab is 3.6.2. Jupyter lab starts for me won’t let me type but a min and if I touch another folder it shuts down. Very slow when it does work.
Here is what my terminal says:
➜ data-analysis source .venv/bin/activate
(.venv) ➜ data-analysis jupyter lab
[I 2023-12-04 05:50:50.660 ServerApp] Package jupyterlab took 0.0000s to import
[I 2023-12-04 05:50:50.665 ServerApp] Package jupyter_server_fileid took 0.0045s to import
[I 2023-12-04 05:50:50.675 ServerApp] Package jupyter_server_terminals took 0.0097s to import
[I 2023-12-04 05:50:50.724 ServerApp] Package jupyter_server_ydoc took 0.0484s to import
[I 2023-12-04 05:50:50.724 ServerApp] Package nbclassic took 0.0000s to import
[W 2023-12-04 05:50:50.727 ServerApp] A _jupyter_server_extension_points function was not found in nbclassic. Instead, a _jupyter_server_extension_paths function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-12-04 05:50:50.727 ServerApp] Package notebook_shim took 0.0000s to import
[W 2023-12-04 05:50:50.727 ServerApp] A _jupyter_server_extension_points function was not found in notebook_shim. Instead, a _jupyter_server_extension_paths function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
[I 2023-12-04 05:50:50.732 ServerApp] jupyter_server_fileid | extension was successfully linked.
[I 2023-12-04 05:50:50.735 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2023-12-04 05:50:50.739 ServerApp] jupyter_server_ydoc | extension was successfully linked.
[I 2023-12-04 05:50:50.743 ServerApp] jupyterlab | extension was successfully linked.
[I 2023-12-04 05:50:50.747 ServerApp] nbclassic | extension was successfully linked.
[I 2023-12-04 05:50:51.101 ServerApp] notebook_shim | extension was successfully linked.
[I 2023-12-04 05:50:51.428 ServerApp] notebook_shim | extension was successfully loaded.
[I 2023-12-04 05:50:51.428 FileIdExtension] Configured File ID manager: ArbitraryFileIdManager
[I 2023-12-04 05:50:51.428 FileIdExtension]
[I 2023-12-04 05:50:51.429 FileIdExtension]
[I 2023-12-04 05:50:51.431 FileIdExtension] ArbitraryFileIdManager : Successfully connected to database file.
[I 2023-12-04 05:50:51.432 FileIdExtension] ArbitraryFileIdManager : Creating File ID tables and indices with journal_mode = DELETE
[I 2023-12-04 05:50:51.435 FileIdExtension] Attached event listeners.
[I 2023-12-04 05:50:51.435 ServerApp] jupyter_server_fileid | extension was successfully loaded.
[I 2023-12-04 05:50:51.437 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2023-12-04 05:50:51.437 ServerApp] jupyter_server_ydoc | extension was successfully loaded.
[I 2023-12-04 05:50:51.438 LabApp] JupyterLab extension loaded from analysis/.venv/lib/python3.11/site-packages/jupyterlab
[I 2023-12-04 05:50:51.438 LabApp] JupyterLab application directory is /Users//Desktop/Python/data-analysis/.venv/share/jupyter/lab
[I 2023-12-04 05:50:51.441 ServerApp] jupyterlab | extension was successfully loaded.
[I 2023-12-04 05:50:51.448 ServerApp] nbclassic | extension was successfully loaded.
[I 2023-12-04 05:50:51.449 ServerApp] Serving notebooks from local directory:

0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[I 2023-12-04 05:50:55.257 LabApp] Build is up to date
[W 2023-12-04 05:50:56.558 ServerApp] SSL Error on 22 (‘2606:50c0:8000::153’, 443, 0, 0): [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)
[I

21

FYI:

I would appreciate if anyone affected, especially Windows users, could confirm that adding "-Xfrozen_modules=off", as a second argument of argv list in kernel.json of their kernelspec (run jupyter kernelspec list from terminal to find where these are located) solves the problem and does not introduce any problems.

For example, the change could look like:

{
 "argv": [
  "python",
+ "-Xfrozen_modules=off",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3 (ipykernel)",
 "language": "python",
 "metadata": {
  "debugger": true
 }
}