Build failed using pipenv

Binder
1

I have a notebook available in a git repo here (5d95b). The repository includes both Pipfile and Pipfile.lock which I understand should be supported.

For a reason that I don’t understand, I’m unable to build the Binder app based on this repo. I believe that the error log is this (full log in this gist):

 ---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
 ---> Running in 0fa756bef166
Installing dependencies from Pipfile.lock (6a71d9)…
Ignoring appnope: markers 'platform_system == "Darwin"' don't match your environment
Ignoring ipython: markers 'python_version >= "3.8"' don't match your environment
Ignoring numpy: markers 'python_version >= "3.10"' don't match your environment
e[91mAn error occurred while installing pandas==1.4.2 --hash=sha256:0010771bd9223f7afe5f051eb47c4a49534345dfa144f2f5470b27189a4dd3b5 --hash=sha256:061609334a8182ab500a90fe66d46f6f387de62d3a9cb9aa7e62e3146c712167 --hash=sha256:09d8be7dd9e1c4c98224c4dfe8abd60d145d934e9fc1f5f411266308ae683e6a --hash=sha256:295872bf1a09758aba199992c3ecde455f01caf32266d50abc1a073e828a7b9d --hash=sha256:3228198333dd13c90b6434ddf61aa6d57deaca98cf7b654f4ad68a2db84f8cfe --hash=sha256:385c52e85aaa8ea6a4c600a9b2821181a51f8be0aee3af6f2dcb41dafc4fc1d0 --hash=sha256:51649ef604a945f781105a6d2ecf88db7da0f4868ac5d45c51cb66081c4d9c73 --hash=sha256:5586cc95692564b441f4747c47c8a9746792e87b40a4680a2feb7794defb1ce3 --hash=sha256:5a206afa84ed20e07603f50d22b5f0db3fb556486d8c2462d8bc364831a4b417 --hash=sha256:5b79af3a69e5175c6fa7b4e046b21a646c8b74e92c6581a9d825687d92071b51 --hash=sha256:5c54ea4ef3823108cd4ec7fb27ccba4c3a775e0f83e39c5e17f5094cb17748bc --hash=sha256:8c5bf555b6b0075294b73965adaafb39cf71c312e38c5935c93d78f41c19828a --hash=sha256:92bc1fc585f1463ca827b45535957815b7deb218c549b7c18402c322c7549a12 --hash=sha256:95c1e422ced0199cf4a34385ff124b69412c4bc912011ce895582bee620dfcaa --hash=sha256:b8134651258bce418cb79c71adeff0a44090c98d955f6953168ba16cc285d9f7 --hash=sha256:be67c782c4f1b1f24c2f16a157e12c2693fd510f8df18e3287c77f33d124ed07 --hash=sha256:c072c7f06b9242c855ed8021ff970c0e8f8b10b35e2640c657d2a541c5950f59 --hash=sha256:d0d4f13e4be7ce89d7057a786023c461dd9370040bdb5efa0a7fe76b556867a0 --hash=sha256:df82739e00bb6daf4bba4479a40f38c718b598a84654cbd8bb498fd6b0aa8c16 --hash=sha256:f549097993744ff8c41b5e8f2f0d3cbfaabe89b4ae32c8c08ead6cc535b80139 --hash=sha256:ff08a14ef21d94cdf18eef7c569d66f2e24e0bc89350bcd7d243dd804e3b5eb2! Will try again.
e[0mWARNING: Ignoring invalid distribution -bformat (/srv/conda/envs/notebook/lib/python3.7/site-packages)
Collecting pygments==2.13.0
  Downloading Pygments-2.13.0-py3-none-any.whl (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 4.3 MB/s eta 0:00:00
Installing collected packages: pygments
  Attempting uninstall: pygments
    Found existing installation: Pygments 2.12.0
    Uninstalling Pygments-2.12.0:
      Successfully uninstalled Pygments-2.12.0
Successfully installed pygments-2.13.0
WARNING: Ignoring invalid distribution -bformat (/srv/conda/envs/notebook/lib/python3.7/site-packages)
Requirement already satisfied: pyrsistent==0.18.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from -r /tmp/pipenv-l1bf6btw-requirements/pipenv-a7v98uyc-requirement.txt (line 1)) (0.18.1)
Collecting pyzmq==23.2.1
  Downloading pyzmq-23.2.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 7.8 MB/s eta 0:00:00
Installing collected packages: pyzmq
  Attempting uninstall: pyzmq
    Found existing installation: pyzmq 23.2.0
    Uninstalling pyzmq-23.2.0:
      Successfully uninstalled pyzmq-23.2.0
WARNING: Ignoring invalid distribution -ytz (/srv/conda/envs/notebook/lib/python3.7/site-packages)
Successfully installed pyzmq-23.2.1
Installing initially failed dependencies…
e[91m[pipenv.exceptions.InstallError]:   File "/srv/conda/envs/notebook/lib/python3.7/site-packages/pipenv/core.py", line 1874, in do_install
e[0me[91m[pipenv.exceptions.InstallError]:       keep_outdated=keep_outdated
e[0me[91m[pipenv.exceptions.InstallError]:   File "/srv/conda/envs/notebook/lib/python3.7/site-packages/pipenv/core.py", line 1253, in do_init
e[0me[91m[pipenv.exceptions.InstallError]:       pypi_mirror=pypi_mirror,
e[0me[91m[pipenv.exceptions.InstallError]:   File "/srv/conda/envs/notebook/lib/python3.7/site-packages/pipenv/core.py", line 859, in do_install_dependencies
e[0me[91m[pipenv.exceptions.InstallError]:       retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
e[0me[91m[pipenv.exceptions.InstallError]:   File "/srv/conda/envs/notebook/lib/python3.7/site-packages/pipenv/core.py", line 763, in batch_install
e[0me[91m[pipenv.exceptions.InstallError]:       _cleanup_procs(procs, not blocking, failed_deps_queue, retry=retry)
e[0me[91m[pipenv.exceptions.InstallError]:   File "/srv/conda/envs/notebook/lib/python3.7/site-packages/pipenv/core.py", line 681, in _cleanup_procs
e[0me[91m[pipenv.exceptions.InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
e[0me[91m[pipenv.exceptions.InstallError]: []
e[0me[91m[pipenv.exceptions.InstallError]: ['ERROR: Ignored the following versions that require a different python version: 1.4.0 Requires-Python >=3.8; 1.4.0rc0 Requires-Python >=3.8; 1.4.1 Requires-Python >=3.8; 1.4.2 Requires-Python >=3.8; 1.4.3 Requires-Python >=3.8', 'ERROR: Could not find a version that satisfies the requirement pandas==1.4.2 (from versions: 0.1, 0.2, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.16.2, 0.17.0, 0.17.1, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.21.1, 0.22.0, 0.23.0, 0.23.1, 0.23.2, 0.23.3, 0.23.4, 0.24.0, 0.24.1, 0.24.2, 0.25.0, 0.25.1, 0.25.2, 0.25.3, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5)', 'ERROR: No matching distribution found for pandas==1.4.2']
e[0me[91mERROR: ERROR: Package installation failed...
e[0mRemoving intermediate container 0fa756bef166
The command '/bin/sh -c (cd . &&                     PATH="${KERNEL_PYTHON_PREFIX}/bin:$PATH"                         pipenv install --ignore-pipfile --system --dev                 )' returned a non-zero code: 1Built image, launching...
Failed to connect to event stream

From the logs I understand that there is a mismatch between:

  • The Python version I’m using in the environment (3.10) and the one used when building the image on Binder’s side.
  • The platform (Darwin/Mac) vs. probably some NIX.

How can I solve the problem? I’m happy to provide more details.

2

repo2docker is the underlying tool that builds images on mybinder.org
According to the source code
repo2docker/__init__.py at d415115b66f24d4165842f94a892987959c9cde3 · jupyterhub/repo2docker · GitHub
it should detect the required Python version from your Pipfile.lock.

Could you try installing the latest version of repo2docker (from the git main branch) locally and see if you still get an error?

If you do, please could you create a minimum reproducible example, and open a bug report on that repo? Thanks!

3

I believe this was caused by the lack of a Python 3.10 env in repo2docker: add Python 3.10 base environment by minrk · Pull Request #1175 · jupyterhub/repo2docker · GitHub which resulted in using a default Python version (3.7), which was incompatible with the pinned dependencies in Pipfile.lock.

If you specify Python 3.9 or wait for the linked PR to be deployed, I think it should work.

1 Like
4

@minrk comments seems to be in the right direction. In this branch I downgraded Python to version 3.9 and there was progress.

I’m able to launch Jupyter lab and open the notebook. However, when trying to run the first cell I get an error ImportError: Unable to import required dependencies: numpy: No module named 'numpy'.

Probably there was some issue during the image building, but no, as it is already built, I don’t know how to rebuild it. Is there a way other than changing something in the repo and trying again?

5

I’m not familir with pipfile/pipenv, but I noticed
"markers": "python_version < '3.10' and platform_machine == 'arm64'", in candy-analysis/Pipfile.lock at d1ad924ee430c32faf7faf5b965788460d9fce64 · drorata/candy-analysis · GitHub

Does that mean numpy will only be installed on arm64 machines?

1 Like
6

Seems like @manics nailed it… See: Multi Platform support for Pipfile.lock · Issue #5130 · pypa/pipenv · GitHub.

I guess trying to move away from conda and picking pipenv wasn’t the best move :frowning:

1 Like
7

I removed the Pipfile.lock and it seems like I was unblocked. See Multi Platform support for Pipfile.lock · Issue #5130 · pypa/pipenv · GitHub for more details.