Creating environment.yml for Python 2 via Binder

Binder repo help
1

The docs for repo2docker contain a helpful section about how to export an environment.yml file with pinned versions using conda: https://repo2docker.readthedocs.io/en/latest/howto/export_environment.html

However, this did not work for me, because my original environment.yml looks like this:

name: pcraster-python-environment
channels:
  - conda-forge
  - defaults
dependencies:
  - python=2.7
  - numpy
  - matplotlib

So, obviously dependencies are missing as this just includes the packages I am aware of, but no transitive deps. When I run conda env export in a terminal, I get the information about the default Python 3 installation (partial output only):

name: root
channels:
  - conda-forge
  - defaults
dependencies:
  - asn1crypto=0.24.0=py37_0
[...]
  - pysocks=1.6.8=py37_0
  - python=3.7.3=h0371630_0
[...]
prefix: /srv/conda

(With the side issue that copying the output did not work as described, solved that by outputting to a file.)

Is there a way to tell conda to give me the environment for Python 2?

What I did do is adding a cell to my notebook running the Python 2 kernel with the following code and output, which looks like what I probably want (partial output only):

from pip._internal.operations.freeze import freeze
for requirement in freeze(local_only=True):
    print(requirement)

backports-abc==0.5
backports.functools-lru-cache==1.5
backports.shutil-get-terminal-size==1.0.0
certifi==2019.6.16
[..]
pyparsing==2.4.0
Python==2.7.15
python-dateutil==2.8.0
[...]

Binder to try this all out at https://mybinder.org/v2/gh/nuest/PLUC_Mozambique/master?urlpath=%2Flab%2Ftree%2Fmodel%2FPLUC.ipynb

2

Short update: When I try to use all versions found the above way in environment.yml I get the a build error:

image

3

Which conda environment was active when you ran conda env export and where did you execute that command? If this was in a repo2docker container then the chances are that the env with Python 2 in it wasn’t activated. This is a known bug waiting for someone to fix it https://github.com/jupyter/repo2docker/issues/698 (this was also reported from the Turing way project but I can’t locate the issue. Do @choldgraf or @sgibson91 remember where this is?)

4

It’s this one :slight_smile:

1 Like
5

Thanks for pointing to the issues. This might all be expected behaviour, but I’m not a conda expert, so documenting some steps here:

  1. Taking the hint from @sgibson91 using source activate kernel in the terminal I get the correct version:

image
image.png728Ă—240 19 KB

[…]
image
image.png744Ă—239 15.5 KB

But as you can see the root environment is still Python 3.x.

If I specify kernel, then I get what I think I want:

image
image.png732Ă—158 9.65 KB

[…]
image
image.png778Ă—447 32.7 KB

Should the instructions for pinning the version be updated to suggest using kernel instead of root ?

So, just noting here the way to dump the environment for different installed environments here for the next person to stumble upon this without proper conda knowledge.

I’ll take the other discussion to GitHub in the issues linked above.

6

Yes. Well, you should use the kernel env if it exists. Currently it exists when the user needs Python 2, otherwise there is no extra environment.

If you could make a PR to fix up the docs that would be great to help turn it from “known problem” to something that doesn’t trip up the next person :slight_smile:

7

With https://github.com/jupyter/repo2docker/pull/737 I suggest to drop the named environment completely, because the desired version is activated in the terminal automatically. Is that feasible?

8

yeah, almost. We want that to happen all the time except when launching the notebook process itself. that’s the tricky bit.

9

So, could you in that case (and only then) explicitly activate the notebook environment?

10

It’s tricky, because we want the notebook process to start in one env, but then subprocesses launched by that process to activate a different env. One way is to do it explicitly, by making sure there’s a profile script that activates the env, like the one we are using now, then activate the notebook env by hand for only the initial entrypoint.

11

OK - so the PR is not feasible as is :slight_smile:

Should the notebook process should be started with Python 3 always?

Is conda/__init__.py lacking a conda activate notebook somewhere?