Repo2docker: conda and 10 mins limit on travis

Binder
1

We are using repo2docker to build “JupyterHub ready” docker images. Our environment.yml is rather long, but not extremely complex (very few pins).

More often than not, we hit the 10 mins stalled limit on the now infamous Solving environment: ...working... conda step. (sometimes just trying again would eventually work, but recently we haven’t been able to build at all because of that).

Does someone know if there is a way to let conda spawn “something” while solving stuff so that travis does not think it has gone idle?

[EDIT] the travis_wait directive seems to help. Keeping the post open still, in case someone has the same issue.

(or, even better but probably only a dream: how to make conda faster in our rather complex env?)

Thanks!

1 Like
2

At some point I decided to use pip instead because it was faster than conda. This is definitely a context-dependend decision though!

3

I think travis_wait is the best way to deal with processes which don’t output anything for a long time.

I think pinning more packages to s specific version makes the solving faster because there are less possible options that need to be checked by the solver. I think. In my head at least it makes sense that this would be true because by providing compatible pins you are reducing the search space, so it has to get faster.

I don’t have the links handy but there are a few blog posts by the conda maintainers that tell you how to output timing information. I’ve found it useful to turn that on once in a while to see which step of the conda install run is slow. Sometimes it turned out to be fetching packages, repo data, unpacking packages, etc. All of which take a while and speed ups come mostly from moving to newer versions of conda. Solver stuff can be fixed by pinning or maybe not but again the timing information will tell you.

Maybe @bollwyvl can point you to the information about finding out which part of conda is taking time.

4

Thanks all for your messages!

Yes, it did help! Thanks a lot for that, I never thought of that

Yes, we might try that eventually as well. Given our dependencies it might be tricky, but then a lot of packages have wheels nowadays, this is helping a lot.

1 Like