Module versions in Python

Could someone explain why in Jupyter Lab and Notebood that Python and it’s modules are so well behind current versions. For example Python 3.9 by default when 3.10 has been around for more than a year and 3.11 is now appearing; Pandas 1.4.4 when 1.5.2 is now available and widely used. My concern is that newer versions have big fixes, improved performance and additional useful features. I appreciate there may well be resource issues but I don’t understand what the technical issues are in bringing-in newer versions.

What is the context here? How do you install and run Jupyter tools?

This sounds like it might be an issue with your provider of these packages, and not one of the volunteer community that creates official distributions.

We try really hard to keep the jupyter stack working with the the oldest supported (3.7) up to the latest Python, with a number of projects testing against python’s nightly dev branch.

Wherever possible, the Jupyter python packages are distributed on PyPI as py3-none-any.whl files, with unbounded top python_requires, so there’s usually 0 lag when a new python release drops. There are some newer jupyter_server packages that depend on rust build chains, so they will be harder to build, especially for exotic/proprietary platforms, like Apple Silicon.

On conda-forge, there can sometimes be a delay, for example if a particular compiler version changes, but usually we get up to a working jupyterlab within the first day or two after an official release.

Linux distributions lag behind, as they are generally more conservative.

2 Likes

Thanks for the response. I am using Anaconda on MS Windows. I installed recently and simply accepted the defaults. Any advice on keeping up to date would be very welcome.

That might be something you could reach out to anaconda through their support channels.

Otherwise, on windows, I’d recommend:

  • stay within the conda ecosystem
    • but don’t use anaconda
      • and in fact uninstall it, and remove any modifications it made your machine, such as environment variables
  • start with a clean mambaforge
    • this is made of all conda-forge packages, and makes conda-forge your default channel
    • put it in a humorously short path, e.g. C:\mf
      • but actually read all the options
        • don’t apply any shell modifications
          • e.g. add to path, register as python
  • use the provided “prompt” desktop shortcut to start it
    • windows terminal (from the windows store) with cmd.bat is highly recommended
  • in that base env, only ever install/keep up-to-date mamba
    • and its dependencies (e.g. python)
  • for each other project you work on (or class, or client, or whatever)
    • make (and check in to version control) an environment.yml that describes the “leaf” dependencies,
    • e.g.
      • python >=3.11,<3.12
      • pandas >=1.5.2,<2
      • jupyterlab >=3.5.2,<4
    • create those environments with mamba env update -n project-1 --file environment.yml
      • indeeed, only ever use mamba env update, and add new dependencies there
        • package-by-package installation is much slower, and more likely to get stuck in strange local minima vs solving the whole env
    • activate them when using them
      • you can make more Windows Terminal shortcuts

With this setup:

  • it’s far less likely to end up in a place where you can’t update mamba itself
    • and it can do that more quickly than conda
  • your individual environments are separate, and can be individually updated without the risk of breaking everything
  • you
2 Likes