# Does mybinder limit background processes? Differing behavior between mybinder and repo2docker

**URL:** <https://discourse.jupyter.org/t/does-mybinder-limit-background-processes-differing-behavior-between-mybinder-and-repo2docker/9855>\
**Category:** repo help\
**Tags:** help-wanted\
**Created:** [July 4, 2021, 6:50am UTC](https://discourse.jupyter.org/t/does-mybinder-limit-background-processes-differing-behavior-between-mybinder-and-repo2docker/9855 "2021-07-04T06:50:16Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![matthewfeickert](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/matthewfeickert/32/70_2.png) [@matthewfeickert](https://discourse.jupyter.org/u/matthewfeickert)\
**Post date:** [July 4, 2021, 6:50am UTC](https://discourse.jupyter.org/t/does-mybinder-limit-background-processes-differing-behavior-between-mybinder-and-repo2docker/9855/1 "2021-07-04T06:50:16Z")

</div>

Hi. I’m getting different runtime behavior between running a Binderized repo on the public [`mybinder`](https://mybinder.org/) instance and running it locally on my machine with `repo2docker`. Does `mybinder` limit the ability to have background processes run in parallel to the Jupyter server?

**reproducible minimal failing / working example:** I’m writing a notebook talk for an example at a conference [GitHub - pyhf/pyhep-2021-notebook-talk: Jupyter notebook talk given at PyHEP 2021 on distributed statistical inference with pyhf](https://github.com/pyhf/pyhep-2021-notebook-talk). If you run the Binderized instance of the draft example on the current commit of `f3fd846e9a24cf8c0f148e43ca033a8cecb92012` (c.f. launch button)

[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/pyhf/pyhep-2021-notebook-talk/f3fd846e9a24cf8c0f148e43ca033a8cecb92012?urlpath=lab/tree/talk.ipynb)

and go through the necessary authentication with Globus and then run the example on the local host you will get the following error for the cell

```python
# retrieve output
result = fxc.get_result(task_id)

```

```auto
---------------------------------------------------------------------------
TaskPending Traceback (most recent call last)
<ipython-input-13-ddf3088a7076> in <module>
      1 # retrieve output
----> 2 result = fxc.get_result(task_id)

/srv/conda/envs/notebook/lib/python3.7/site-packages/funcx/sdk/client.py in get_result(self, task_id)
    250 task = self.get_task(task_id)
    251 if task['pending'] is True:
--> 252 raise TaskPending(task['status'])
    253 else:
    254 if 'result' in task:

TaskPending: Task is pending due to {self.reason}

```

Normally this would mean that the underlying tool ([`funcX`](https://funcx.readthedocs.io/)) hasn’t finished executing the job yet, but the test case is designed to be trivial to compute and so should be done in well under a second of runtime. It looks like from inspection

```auto
$ ps -u $USER
    PID TTY TIME CMD
      1 ? 00:00:00 python3
     18 ? 00:00:07 jupyter-noteboo
     84 pts/0 00:00:00 bash
    123 ? 00:00:00 off_process_che <defunct>
    124 ? 00:00:00 funcx-endpoint <defunct>
    125 ? 00:00:20 funcx-endpoint
    128 ? 00:00:00 off_process_che
    139 ? 00:00:59 funcx-endpoint
    142 ? 00:00:00 off_process_che
    146 ? 00:00:00 bash <defunct>
    147 ? 00:00:00 bash <defunct>
    164 ? 00:00:00 bash <defunct>
    165 ? 00:00:01 funcx-manager <defunct>
    175 ? 00:00:00 funcx-worker <defunct>
    224 ? 00:00:00 off_process_che <defunct>
   2117 pts/0 00:00:00 ps

```

that the `funcx-worker`s are having problems as well. Does `mybinder` forbid additional processes?

If I build on the same commit with `repo2docker` though

```auto
$ repo2docker --ref f3fd846e9a24cf8c0f148e43ca033a8cecb92012 https://github.com/pyhf/pyhep-2021-notebook-talk

```

and run through the same process it is able to run with no issues

```auto
# retrieve output
>>> result = fxc.get_result(task_id)
>>> result
{'cls_obs': 0.0457153916230104, 'fit-time': 0.06638431549072266}

```

Any thoughts on what might be the difference in behavior between the two?

---

<div class="post-metadata">

**Author:** ![manics](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/manics/32/85_2.png) [@manics](https://discourse.jupyter.org/u/manics)\
**Post date:** [July 4, 2021, 11:54am UTC](https://discourse.jupyter.org/t/does-mybinder-limit-background-processes-differing-behavior-between-mybinder-and-repo2docker/9855/2 "2021-07-04T11:54:03Z")

</div>

Main differences I can think of are:

- [limited outbound ports](https://github.com/jupyterhub/mybinder.org-deploy/blob/5af8d5726348e46534f3e50273c082a6aa75889b/mybinder/values.yaml#L52-L58)
- [no privilege escalation](https://github.com/jupyterhub/mybinder.org-deploy/blob/5af8d5726348e46534f3e50273c082a6aa75889b/mybinder/values.yaml#L224)
- [limited resources](https://github.com/jupyterhub/mybinder.org-deploy/blob/5af8d5726348e46534f3e50273c082a6aa75889b/mybinder/values.yaml#L282-L287)

I can reproduce your error on mybinder. Is there a way to get the funcx daemon to output some logs?

---

<div class="post-metadata">

**Author:** ![matthewfeickert](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/matthewfeickert/32/70_2.png) [@matthewfeickert](https://discourse.jupyter.org/u/matthewfeickert)\
**Post date:** [July 4, 2021, 3:03pm UTC](https://discourse.jupyter.org/t/does-mybinder-limit-background-processes-differing-behavior-between-mybinder-and-repo2docker/9855/3 "2021-07-04T15:03:01Z")

</div>

> Is there a way to get the funcx daemon to output some logs?

I’ll [ask the devs](https://github.com/funcx-faas/funcX/issues/543) (there will be a delay from them as there is a US holiday) but [from the docs](https://funcx.readthedocs.io/en/latest/debugging.html) it looks like it might require setting up a redis server and then altering the funcX config to point to it.

---

<div class="post-metadata">

**Author:** ![manics](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/manics/32/85_2.png) [@manics](https://discourse.jupyter.org/u/manics)\
**Post date:** [July 4, 2021, 4:02pm UTC](https://discourse.jupyter.org/t/does-mybinder-limit-background-processes-differing-behavior-between-mybinder-and-repo2docker/9855/4 "2021-07-04T16:02:05Z")

</div>

From [Globus Compute Endpoints - Globus Compute 2.8.0 documentation](https://funcx.readthedocs.io/en/latest/endpoints.html#starting-an-endpoint)

> The endpoint establishes three outbound ZeroMQ channels to the forwarder (on the three ports returned during registration) to retrieve tasks, send results, and communicate command information.

This _might_ be the problem- if those are three random ports they’ll be blocked.