Hub Error about SQLite3 Version

JupyterHub Zero to JupyterHub on Kubernetes
1

Hi, there,

I rebuild Hub docker image based on amazon linux2. When I tried to run it locally, I got error:
For the sqlite version error: sqlalchemy.exc.NotSupportedError: (sqlite3.NotSupportedError) deterministic=True requires SQLite 3.8.3 or higher

The default SQLite coming with amazon linux2 is 3.7.17. However, I have upgrade it to the latest version (3.40.x). I have deleted old sqlite from /usr/bin and create symbol link to /usr/local/bin (where the new version is installed). However, I still get above error.

I don’t know from where the JupyterHub picks up the SQLite version? And how to fix?

Thanks for any help!

2

This is much more a question about linux distribution and packaging than it is z2jh.

In the other discussion you said you had a policy to use Amazon Linux. My first question is could you use Amazon Linux 2022 that’s based off Fedora rather than the very dated Centos 7

In your Dockerfile use

FROM amazonlinux:2022

Doing the below shows the much more modern python and sqlite which should make things easier

docker pull amazonlinux:2022
docker run -it --rm amazonlinux:2022 rpm -q sqlite-libs pyt
hon3
sqlite-libs-3.40.0-1.amzn2022.0.1.x86_64
python3-3.9.16-1.amzn2022.0.1.x86_64

In response to your original question, there is a difference between sqlite binary and a library - the python sqlite bindings are compiled against the system sqlite when python is built (ie at rpmbuild). Thus your python doesn’t use /usr/bin/sqlite3 but the library as can be seen via ldd of the _sqlite3 module.

amazon-linux-extras install python3.8
rpm -ql python38-libs | grep _sqlite
/usr/lib64/python3.8/lib-dynload/_sqlite3.cpython-38-x86_64-linux-gnu.so
ldd /usr/lib64/python3.8/lib-dynload/_sqlite3.cpython-38-x86_64-linux-gnu.so
	linux-vdso.so.1 (0x00007ffc6b3fa000)
	libsqlite3.so.0 => /lib64/libsqlite3.so.0 (0x00007f5d37918000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5d376fa000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f5d3734d000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f5d37149000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f5d37de0000)

To get a working python with newer sqlite3 you will need to do the work to build/install or package yourself. You probably don’t want to replace the sqlite as that’ll break the OS image, so would need to install in a different location (or prefix) and build python for that

There are many ways of managing your OS python to support more recent versions, such as building your own RPMS using an alternate prefix (eg /opt or /usr/local), using a python version management tool such a pyenv, or other tools.

One (untested by me) way is to look at how folks running centos are doing this to get an idea of how to setup manually, Amazon Linux 2 is related to CentOS 7 https://www.webdesignsun.com/insights/upgrading-sqlite-on-centos/

Like your previous question, this far exceeds what the Jupyter community can support - z2jh provides containers that are known to work, if you need to build your own due to policies there is only limited support that can be given.

1 Like
3

@pnasrat Thank you so very much for the great help, Pris! You are the best : - )

4

I would also recommend looking at tljh.jupyter.org/ (which requires Ubuntu) instead if you are just trying to set up a single node JupyterHub :slight_smile:

5

@yuvipanda Thanks for the information, Yuvi! I will take a look : - )