# How to compile cython files for use in binder?

**URL:** <https://discourse.jupyter.org/t/how-to-compile-cython-files-for-use-in-binder/15140>\
**Category:** Binder\
**Created:** [July 29, 2022, 6:06pm UTC](https://discourse.jupyter.org/t/how-to-compile-cython-files-for-use-in-binder/15140 "2022-07-29T18:06:50Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![tadpoleloop](https://avatars.discourse-cdn.com/v4/letter/t/13edae/32.png) [@tadpoleloop](https://discourse.jupyter.org/u/tadpoleloop)\
**Post date:** [July 29, 2022, 6:06pm UTC](https://discourse.jupyter.org/t/how-to-compile-cython-files-for-use-in-binder/15140/1 "2022-07-29T18:06:50Z")

</div>

I have a github repo with some `.pyx` files. I can compile them from the command line with `python setup.py`

setup.py looks like:

```auto
from distutils.core import setup
from Cython.Build import cythonize
setup(
    ext_modules=cythonize('./*.pyx', annotate = True), 
    script_args=['build_ext'],                                        
    options={'build_ext':{'inplace':True}}
)

```

Right now I am using environment.yml to set up my binder, is there some directive to include in that file? Or another file to include? I would like for binder to spin up the image with the files compiled.

---

<div class="post-metadata">

**Author:** ![bollwyvl](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/bollwyvl/32/904_2.png) [@bollwyvl](https://discourse.jupyter.org/u/bollwyvl)\
**Post date:** [July 30, 2022, 2:00pm UTC](https://discourse.jupyter.org/t/how-to-compile-cython-files-for-use-in-binder/15140/2 "2022-07-30T14:00:59Z")

</div>

A [`postBuild`](https://repo2docker.readthedocs.io/en/latest/config_files.html#postbuild-run-code-after-installing-the-environment) file might be the simplest approach. It runs after the environment has been updated, so `pip install -e .`, for example, would work, provided `cython` (and any other build dependencies) are included in the `environment.yml`.

Of note: transitioning to `setuptools` is recommended over `distutils` by the cython docs… and the maintainers of `distutils`.

---

<div class="post-metadata">

**Author:** ![tadpoleloop](https://avatars.discourse-cdn.com/v4/letter/t/13edae/32.png) [@tadpoleloop](https://discourse.jupyter.org/u/tadpoleloop)\
**Post date:** [July 30, 2022, 2:49pm UTC](https://discourse.jupyter.org/t/how-to-compile-cython-files-for-use-in-binder/15140/3 "2022-07-30T14:49:32Z")

</div>

> Of note: transitioning to `setuptools` is recommended over `distutils` by the cython docs… and the maintainers of `distutils` .

Interesting, I didn’t know that, thank you.

Would adding

```auto
- pip:
  - .

```

To the environment.yml file work? As in, will binder run `pip install .`?

---

<div class="post-metadata">

**Author:** ![bollwyvl](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/bollwyvl/32/904_2.png) [@bollwyvl](https://discourse.jupyter.org/u/bollwyvl)\
**Post date:** [July 30, 2022, 3:06pm UTC](https://discourse.jupyter.org/t/how-to-compile-cython-files-for-use-in-binder/15140/4 "2022-07-30T15:06:39Z")

</div>

Yes, that `pip` trick works.

From an entropy/cache perspective: in binder (well, docker in general), I don’t recommend doing anything “fancy” with `environment.yml`, as it triggers an [undesirable code path](https://github.com/jupyterhub/repo2docker/blob/42f83f0af4e8532f4cf9d719062345a48146d66e/repo2docker/buildpacks/conda/ __init__.py#L238) which forces the environment to be resolved, re-downloaded, and re-linked _every commit_, instead of just when the `environment.yml` changes.

I find having a local- and docker-compatible `Makefile` or `dodo.py` which handles complex, dependency-based environment tasks, which either install or assume the “base” environment, leads to the most reproducible outcomes… especially when coupled with tools like `conda-lock` and `pip-tools` to generate lockfiles, as these are _great_ sources of predictable entropy.

---

<div class="post-metadata">

**Author:** ![tadpoleloop](https://avatars.discourse-cdn.com/v4/letter/t/13edae/32.png) [@tadpoleloop](https://discourse.jupyter.org/u/tadpoleloop)\
**Post date:** [July 30, 2022, 3:14pm UTC](https://discourse.jupyter.org/t/how-to-compile-cython-files-for-use-in-binder/15140/5 "2022-07-30T15:14:29Z")

</div>

So, in summary, I should instead have a file called `postBuild` and in it should just be the line `pip install -e .`?

And forget doing anything in environment.yml that isn’t declaring simple dependencies?

---

<div class="post-metadata">

**Author:** ![bollwyvl](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/bollwyvl/32/904_2.png) [@bollwyvl](https://discourse.jupyter.org/u/bollwyvl)\
**Post date:** [July 30, 2022, 5:25pm UTC](https://discourse.jupyter.org/t/how-to-compile-cython-files-for-use-in-binder/15140/6 "2022-07-30T17:25:35Z")

</div>

Right, a cache-friendly, defensive setup might be:

- `environment.yml`

```auto
channels:
  - conda-forge
  - nodefaults
dependencies:
  - cython # and any other build deps
  - python >=3.9,<3.10 # or whatever version
  - jupyterlab >=3.4.4,<4 # or whatever version
  - pip
  - pip:
    - any-deps-not-met-by-conda

```

- `postBuild`

```auto
#!/usr/bin/env bash
python -m pip install -vv --no-deps --ignore-installed -e .
python -m pip list # so you have info for...
python -m pip check # ...catching packaging issues during build with log

```