MP Workshop on MyBinder

Thanks for putting in the effort to figure out if mybinder.org is for you and then posting about it!

I think the most important thing to remember is that mybinder.org is a shared resource and operated entirely by volunteers. This means several other people might decide to run large workshops at the same time (unlikely but you never know when you will get slashdotted) or decide to do evil stuff to it. The operators are distributed across Europe and the US, so we have good timezone coverage but it might still take 1-2h for someone to have time to react if something should go wrong. While total outages have become rare (touch wood), it is a good idea to have a backup plan!

Launch times (when the image is built) are determined largely by how big your image is. We will have to transfer it from our registry to the node(s) that launch it. If your image is 700MB this will take a lot less time than if it is 7GB. I had a quick look at your repository and you use a custom Dockerfile. This means your image is unlikely to share layers with other images (already present on the node) and the base image will have to be fetched from docker hub (slower than our local registry and it is a large, rarely used base image).

You install a lot of stuff, if your sessions are 75min will you need all of it? I’d invest some time to check if you can do most of what you do manually in the Dockerfile with the config files https://repo2docker.readthedocs.io/ recognises. Maybe you can reduce the number of dependencies/software/data you include as well as not everything will be covered? All in an attempt to shrink the image and increase layer sharing.

The timeout is 10-20min of inactivity, there is unfortunately not much that can be done about that, except to not be idle :wink: (that or find a donor who wants to massively increase the compute resources we have :wink: ). I find https://github.com/data-8/nbzip is a good extension to install that allows people to easily download the state of their work to their laptops before breaking for coffee. (In the long term there are plans for allowing people to push their work back to GitHub but no one has started work on that, …yet (hinthintwinkwink).)

Some other tips for image building:

  • make a tag for the commit you are going to use for the workshop
  • make a mybinder.org link that points to this particular tag and point people to that (not master)
  • make sure to build that image at least once so when your workshop starts it has already been built

For outgoing network traffic: talk to the owner of the API to make sure you won’t get rate limited as all requests will appear to come from the same IP.