Pass parameters to a notebook on jupyterhub/binderhub

I’ve been brainstorming with a few others on ways to link JupyterHub/BinderHub with a web-based data repository (in this case https://idr.openmicroscopy.org/ though the pattern is useful for other repositories). For example:

  1. User finds a dataset or item of interest in the repository
  2. That item is associated with one or more recommended notebooks.
  3. User clicks a mybinder.org link associated with the item that launches the notebook, configured to run the analysis on the selected item. In the case of IDR this link would be in the right-hand pane of the dataset browser along with other metadata associated with the dataset or image.

Assume the notebook in step 3 is generic and can work with multiple datasets/items- e.g. a notebook to calculate image statistics, or to fetch metadata, can be run on any image. It therefore needs a parameter identifying the image to analyse.

A user can obviously edit the parameter, or jupyter-widgets can be used to request input, but this requires the user to copy and paste an ID from the data browser to the notebook in mybinder.

What would be really nice is to have a way to pass those parameters as a URL query. For example, instead of

https://mybinder.org/v2/gist/manics/421f2927bb1dbdbc00754a7669eb3f69/master?filepath=idr-image.ipynb

pass the image id as:

https://mybinder.org/v2/gist/manics/421f2927bb1dbdbc00754a7669eb3f69/master?filepath=idr-image.ipynb&IMAGE_ID=2959331

or perhaps as json to avoid clashing with other params:

https://mybinder.org/v2/gist/manics/421f2927bb1dbdbc00754a7669eb3f69/master?filepath=idr-image.ipynb&notebookparams={"IMAGE_ID":2959331}

These links would be auto-generated for each item.

It’s partly related to what Papermill can do on the CLI, but in the browser. Is the best way to implement this a Jupyter notebook/lab extension, or are there better ways?

There is a bunch of thoughts on a related but different topic in https://github.com/jupyterhub/binderhub/issues/712 which also has a lot of links/issues referring to it.

If I understand you correctly you want to add parameters for the notebook, not the spawner (or BinderHub itself). This is what makes it a bit different from the issue I linked.

There are (at least?) two things to solve:

  • recognise that the query parameters aren’t meant for the spawner and pass them on
  • use a notebook to extract them from the final URL

No good idea right now but wanted to link the existing discussion.


One exciting but likely impractical idea: run a git repository server where https://gitrepos.idr.example.com/<datasetID> spits out a repo that is the same for all datasets except for the notebooks that it contains and what the dataset parameters are set to in the notebooks. Then your BinderHub link would refer to the repo for that dataset. (I have no idea if you can generate git repositories “on the fly” … that is why it is exciting and impractical :slight_smile: )

2 Likes

I managed to bodge a plugin based on nbgitpuller:

If you go to the app /urlparams and pass two query parameters paramsfile=config.json and params={"a":1,"b":false} it writes the file, then redirects you to urlpath

Example on mybinder:

Double escpaing the query params for BinderHub makes this difficult to hand-craft URLs, it would be nice if BinderHub could process just the parameters it recognises and pass the remainder unchanged to the app. nbgitpuller has the same problem.

Another snippet on this topic posted by @minrk on gitter earlier today:

Try: https://notebooks.gesis.org/binder/v2/gh/gesiscss/smm_demo/master?urlpath=%2Fnotebooks%2Fpython%2Fpolitician%2Ftwitter.ipynb%3Fparam%3Dvalue

filepath is okay, and usually better, because it encodes the location of the file within the repo, which binder then resolves to a url. This doesn’t, however, allow specifying url-specific things like params, so you have to fallback on urlpath if you want to do url things. Any and all url things should be supported, though. The url for a notebook is /notebooks/ not /tree/, and apparently the redirect from /tree to /notebooks doesn’t preserve params, which is why your link isn’t working.

2 Likes

Thanks! That works very well for Jupyter notebooks and the code looks relatively simple. The autorun=true option is pretty neat too!

I had a quick look at the JupyterLab extension docs (first time I’ve looked into extensions), am I correct in thinking this sort of extension would be quite complicated because the notebook isn’t loaded directly, instead it goes via JupyterLab?

Since a similar topic was mentioned on GitHub I though I’d add an update.

I packaged the ideas here into an extension: https://github.com/manics/jupyter-notebookparams
MyBinder example included :grinning:

4 Likes

@manics This is awesome and much needed so thank you! I think there may be one or two bugs (I am definitely not an expert so hopefully this is not just my own ignorance). I notice in your example you pass two “?” in your URL. I am not sure if two messes up things or if it will just recognize the second ? as an &. The website I am using redirects the user and it recognizes that the URL provided already has a ? and so it appends the URL with an & instead of a second ?. The second bug is that if you pass & instead of %26 the remaining variables are not imported. I have not control over the website I am trying to interact with. Any suggestions on how I can get this to work?

EDIT: I should note that I am using mybinder for my notebook. The mybinder URL has the mybinder.org/stuff?urlpath=myurlpath.ipynb which then gets appended to mybinder.org/stuff?urlpath=myurlpath.ipynb&variable=value.

Also, the value that is added is a string, which won’t that have an issue because they don’t add any " in the url.

Thanks again!

You probably have more control than you realize about the URL since you say you are using MyBinder. In fact the example on the jupyter-notebookparams page uses MyBinder.
If you study the example launch badge URL there (i.e., https://mybinder.org/v2/gh/manics/jupyter-notebookparams/master?urlpath=notebooks%2Fexample.ipynb%3Fa%3D1%26b%3D%22whatever%22%26autorun%3Dtrue) you’ll see there is no second question mark or quotes because the text is ‘escaped’ after the ?urlpath= portion.
If you go to here and then paste in the box on the left the end of the URL the launch badge uses notebooks%2Fexample.ipynb%3Fa%3D1%26b%3D%22whatever%22%26autorun%3Dtrue, and then finally press ‘Decode URL’ button, you’ll see that ‘escaped’ part decodes to notebooks/example.ipynb?a=1&b="whatever"&autorun=true. So for MyBinder use you should be constructing your URL similar to that badge as well. And you can use that same site to encode it for the link you’ll use to launch MyBInder with your notebook. You’ll use the ‘Encode URL’ button to convert that final portion that includes your notebook path and name and the variables you want to set, and place it after the ?urlpath= portion of your typical MyBinder URL.

For example, to pass different parameters, a = 42 and b = 'foobar' to the test notebook, you would construct the URL https://mybinder.org/v2/gh/manics/jupyter-notebookparams/master?urlpath=notebooks%2Fexample.ipynb%3Fa%3D42%26b%3D"foobar"%26autorun%3Dtrue as a URL. Go ahead and click on that link and you’ll see it launches via MyBinder with those settings now.

2 Likes

Thanks for the response! Maybe I should clarify a bit more what I mean about that I don’t have control over a website. I am build a notebook that analyzes some data. In order to get that data, I have to go through OAuth2. So far, I have been the only one using my notebook, and so I go through an arduous process to get authenticated by hand just so I can get my data. I have some friends that would like to use my notebook and so instead of going through my ‘hack’ to get access, I am wanting to host my notebook on mybinder. To do OAuth2, you go to a certain URL and authenticate. After authenticating, it sends you back to my website with code=authentication_code appended at the end.

As far as I understand as I have looked into it more, I do not think this extension will work for me because it doesn’t quite read the parameters and then pass them in again. Instead, you append the URL with a second set of parameters. So only 1 parameter is read and that parameter is the urlpath=[URLpath]?[parameters=values]. The extenstion reads that urlpath takes you to the URLpath and builds the first cell with the extra parameters. Because I don’t have control of the authentication, my added parameters is added as an actual added parameter and not appended to the URLpath.

Maybe I could find a way to bounce the URL in the right format. But I have found a method where I open the authentication in a IFrame and then display the authentication code. The user has to copy and paste it, but that is easy enough to do for my friends. Any suggestions on how to do this better is welcome! I am definitely not a programer, so I am just faking my way through this.

@manics I totally missed this, it is very cool!

2 Likes