# Datalad.api works in terminal ipython but not Jupyter lab; how to fix?

**URL:** <https://discourse.jupyter.org/t/datalad-api-works-in-terminal-ipython-but-not-jupyter-lab-how-to-fix/9218>\
**Category:** JupyterLab\
**Created:** [May 17, 2021, 2:46pm UTC](https://discourse.jupyter.org/t/datalad-api-works-in-terminal-ipython-but-not-jupyter-lab-how-to-fix/9218 "2021-05-17T14:46:01Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![eprzysinda](https://avatars.discourse-cdn.com/v4/letter/e/91b2a8/32.png) [@eprzysinda](https://discourse.jupyter.org/u/eprzysinda)\
**Post date:** [May 17, 2021, 2:46pm UTC](https://discourse.jupyter.org/t/datalad-api-works-in-terminal-ipython-but-not-jupyter-lab-how-to-fix/9218/1 "2021-05-17T14:46:01Z")

</div>

Hi,  
I’m trying to get datalad.api to work in jupyter lab. It seems to work in the terminal with ipython but not within the notebook. Here’s a link to the site with directions on how to use: [http://handbook.datalad.org/en/latest/code\_from\_chapters/usecase\_ml\_code.htm](http://handbook.datalad.org/en/latest/code_from_chapters/usecase_ml_code.htm)

The command that gives me an error in jupyter lab is: import datalad.api as dl

## This is the error I get: /Users/eprzysinda/miniconda3/envs/test/lib/python3.7/site-packages/datalad/cmd.py:375: RuntimeWarning: coroutine ‘run\_async\_cmd’ was never awaited new\_loop = True RuntimeWarning: Enable tracemalloc to get the object allocation traceback

RuntimeError Traceback (most recent call last)  
 in   
----\> 1 import datalad.api as dl

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/ **init**.py in   
46  
47 from .config import ConfigManager  
—\> 48 cfg = ConfigManager()  
49  
50 from .log import lgr

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/config.py in **init** (self, dataset, overrides, source)  
344 self.\_runner = GitWitlessRunner(\*\*run\_kwargs)  
345  
 → 346 self.reload(force=True)  
347  
348 if not ConfigManager.\_checked\_git\_identity:

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/config.py in reload(self, force)  
397 while to\_run:  
398 store\_id, runargs = to\_run.popitem()  
 → 399 self.\_stores[store\_id] = self.\_reload(runargs)  
400  
401 # always update the merged representation, even if we did not reload

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/config.py in \_reload(self, run\_args)  
429 protocol=StdOutErrCapture,  
430 # always expect git-config to output utf-8  
 → 431 encoding=‘utf-8’,  
432 )  
433 store = {}

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/config.py in \_run(self, args, where, reload, \*\*kwargs)  
787 if ‘-l’ in args:  
788 # we are just reading, no need to reload, no need to lock  
 → 789 out = self.\_runner.run(self.\_config\_cmd + args, \*\*kwargs)  
790 return out[‘stdout’], out[‘stderr’]  
791

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/cmd.py in run(self, cmd, protocol, stdin, cwd, env, \*\*kwargs)  
385 protocol\_kwargs=kwargs,  
386 cwd=cwd,  
 → 387 env=env,  
388 )  
389 )

~/miniconda3/envs/test/lib/python3.7/asyncio/base\_events.py in run\_until\_complete(self, future)  
561 “”"  
562 self.\_check\_closed()  
 → 563 self.\_check\_runnung()  
564  
565 new\_task = not futures.isfuture(future)

~/miniconda3/envs/test/lib/python3.7/asyncio/base\_events.py in \_check\_runnung(self)  
524 if events.\_get\_running\_loop() is not None:  
525 raise RuntimeError(  
 → 526 ‘Cannot run the event loop while another loop is running’)  
527  
528 def run\_forever(self):

RuntimeError: Cannot run the event loop while another loop is running

I’m new to jupyter lab and python, so it’s possible I missed something simple when setting things up. Any help is much appreciated.  
Thanks!  
~Emily

---

<div class="post-metadata">

**Author:** ![fomightez](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/fomightez/32/495_2.png) [@fomightez](https://discourse.jupyter.org/u/fomightez)\
**Post date:** [May 17, 2021, 4:15pm UTC](https://discourse.jupyter.org/t/datalad-api-works-in-terminal-ipython-but-not-jupyter-lab-how-to-fix/9218/2 "2021-05-17T16:15:41Z")

</div>

Nope, it is nothing you did or didn’t do. There’s a small incompatibility between some aspects of the current packages. Plus, it isn’t specifically JupyterLab; the related issue arises with the classic notebook interface.  
Fortunately, an easy work-around can be found by going to the GitHub repo and searching the terrn ‘loop’ in the [issues page for the project](https://github.com/datalad/datalad/issues), among the open issues. (I extracted that keyword from the RuntimeError at the end of the traceback you report as a hook to start with. It came up with something that looked relevant easily.)

The post [here](https://github.com/datalad/datalad/issues/5409#issuecomment-773586008) spells out the basics of the work around for this issue. The approach relies on importing `nest_asyncio` and saying you want to apply use of it. It was added to the documentation [here](http://handbook.datalad.org/en/latest/basics/101-135-help.html#asyncio-errors-at-datalad-import) as section ’ 9.4.5.1. asyncio errors at DataLad import’ under ’ Common warnings and error’. I’ll spell out implementing it in steps that will work for novices below.

* * *

## Work-around

1. Open your notebook and add the following code in cell above your normal cells.

```plaintext
%pip install nest_asyncio

```

1. Run that cell and then restart the kernel.

2. Then add another cell towards the top of your notebook and run the following code:

```Python
import nest_asyncio
nest_asyncio.apply()

import datalad.api as dl

```

1. You can choose to save this modified notebook because re-running any of these again is fine.

Note: Once you do the first `%pip install` step in your environment, other notebooks should work **without** requiring that `%pip install` step as long as you add the `import nest_asyncio; nest_asyncio.apply()` lines at the top of your other notebooks before you import datalad.

* * *

Note this issue was cross-posted at StackOverflow as well [here](https://stackoverflow.com/q/67509695/8508004), and I’ve posted a link to this solution [there](https://stackoverflow.com/a/67573586/8508004).

---

<div class="post-metadata">

**Author:** ![eprzysinda](https://avatars.discourse-cdn.com/v4/letter/e/91b2a8/32.png) [@eprzysinda](https://discourse.jupyter.org/u/eprzysinda)\
**Post date:** [May 17, 2021, 7:41pm UTC](https://discourse.jupyter.org/t/datalad-api-works-in-terminal-ipython-but-not-jupyter-lab-how-to-fix/9218/3 "2021-05-17T19:41:25Z")

</div>

Thank you so much for this solution. This allowed for the import and it seems to be working for some datalad code, except for I’m experiencing an error when trying to use git-annex. I’m on a mac so it should have been installed with Datalad according to the instructions: [3. Installation and configuration — The DataLad Handbook](http://handbook.datalad.org/en/latest/intro/installation.html#install)

It seems to be a similar situation where it works in the terminal but not in Jupyter lab and gives an error that git-annex of version \>= 7.20190503 is missing even though it was installed in the terminal environment (git-annex version 8.20210428, conda 4.9.2, python 3.7). Do you think this is an issue that could be solved in a similar way? If not, I’ll make a separate post for this issue. This was also a problem before I fixed the data lad also posted in stack overflow (see above).

## Here are the details of this error: results = ds.status(annex=‘all’); yields

MissingExternalDependency Traceback (most recent call last)  
 in   
----\> 1 results = ds.status(annex=‘all’)

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/distribution/dataset.py in apply\_func(wrapped, instance, args, kwargs)  
501 elif i \>= ds\_index:  
502 kwargs[orig\_pos[i+1]] = args[i]  
 → 503 return f(\*\*kwargs)  
504  
505 setattr(Dataset, name, apply\_func(f))

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/interface/utils.py in eval\_func(wrapped, instance, args, kwargs)  
484 return results  
485 lgr.log(2, “Returning return\_func from eval\_func for %s”, wrapped\_class)  
 → 486 return return\_func(generator\_func)(\*args, \*\*kwargs)  
487  
488 return eval\_func(func)

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/interface/utils.py in return\_func(wrapped\_, instance\_, args\_, kwargs\_)  
472 # unwind generator if there is one, this actually runs  
473 # any processing  
 → 474 results = list(results)  
475 # render summaries  
476 if not result\_xfm and result\_renderer in (‘tailored’, ‘default’):

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/interface/utils.py in generator\_func(\*\_args, \*\*\_kwargs)  
403 result\_log\_level,  
404 # let renderers get to see how a command was called  
 → 405 allkwargs):  
406 for hook, spec in hooks.items():  
407 # run the hooks before we yield the result

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/interface/utils.py in \_process\_results(results, cmd\_class, on\_failure, action\_summary, incomplete\_results, result\_renderer, result\_log\_level, allkwargs)  
559 render\_n\_repetitions = 10 if sys.stdout.isatty() else float(“inf”)  
560  
 → 561 for res in results:  
562 if not res or ‘action’ not in res:  
563 # XXX Yarik has to no clue on how to track the origin of the

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/core/local/status.py in **call** (path, dataset, annex, untracked, recursive, recursion\_limit, eval\_subdataset\_state, report\_filetype)  
426 eval\_subdataset\_state,  
427 report\_filetype == ‘eval’,  
 → 428 content\_info\_cache):  
429 yield dict(  
430 r,

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/core/local/status.py in \_yield\_status(ds, paths, annexinfo, untracked, recursion\_limit, queried, eval\_submodule\_state, eval\_filetype, cache)  
128 init=status,  
129 eval\_availability=annexinfo in (‘availability’, ‘all’),  
 → 130 ref=None)  
131 for path, props in status.items():  
132 cpath = ds.pathobj / path.relative\_to(repo\_path)

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/support/annexrepo.py in get\_content\_annexinfo(self, paths, init, ref, eval\_availability, key\_prefix, \*\*kwargs)  
3215 cmd += [‘–include’, ‘\*’]  
3216  
 → 3217 for j in self.call\_annex\_records(cmd, files=files):  
3218 path = self.pathobj.joinpath(ut.PurePosixPath(j[‘file’]))  
3219 rec = info.get(path, None)

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/support/annexrepo.py in call\_annex\_records(self, args, files)  
1204 See `_call_annex()` for more information on Exceptions.  
1205 “”"  
 → 1206 return self.\_call\_annex\_records(args, files=files)  
1207  
1208 def call\_annex(self, args, files=None):

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/support/annexrepo.py in \_call\_annex\_records(self, args, files, jobs, git\_options, stdin, merge\_annex\_branches, progress, \*\*kwargs)  
1085 stdin=stdin,  
1086 merge\_annex\_branches=merge\_annex\_branches,  
 → 1087 \*\*kwargs,  
1088 )  
1089 except CommandError as e:

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/support/annexrepo.py in \_call\_annex(self, args, files, jobs, protocol, git\_options, stdin, merge\_annex\_branches, \*\*kwargs)  
930 “”"  
931 if self.git\_annex\_version is None:  
 → 932 self.\_check\_git\_annex\_version()  
933  
934 # git portion of the command

~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/support/annexrepo.py in \_check\_git\_annex\_version(cls)  
553 )  
554 if not ver:  
 → 555 raise MissingExternalDependency(\*\*exc\_kwargs)  
556 elif ver \< cls.GIT\_ANNEX\_MIN\_VERSION:  
557 raise OutdatedExternalDependency(ver\_present=ver, \*\*exc\_kwargs)

MissingExternalDependency: git-annex of version \>= 7.20190503 is missing. Visit [Redirection — The DataLad Handbook](http://handbook.datalad.org/r.html?install) for instructions on how to install DataLad and git-annex.

---

<div class="post-metadata">

**Author:** ![fomightez](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/fomightez/32/495_2.png) [@fomightez](https://discourse.jupyter.org/u/fomightez)\
**Post date:** [May 17, 2021, 8:11pm UTC](https://discourse.jupyter.org/t/datalad-api-works-in-terminal-ipython-but-not-jupyter-lab-how-to-fix/9218/4 "2021-05-17T20:11:18Z")

</div>

I’m hoping you can add another cell after step #1 from above and then run the following code before you do the imports:

```nohighlight
%conda install -c conda-forge git-annex

```

Be sure to restart the kernel after running that. This may work because [it is found at conda](https://anaconda.org/conda-forge/git-annex).

I am unable to test this though myself because you don’t provide enough code that could be used to test this in an independent environment.

---

<div class="post-metadata">

**Author:** ![eprzysinda](https://avatars.discourse-cdn.com/v4/letter/e/91b2a8/32.png) [@eprzysinda](https://discourse.jupyter.org/u/eprzysinda)\
**Post date:** [May 17, 2021, 8:51pm UTC](https://discourse.jupyter.org/t/datalad-api-works-in-terminal-ipython-but-not-jupyter-lab-how-to-fix/9218/5 "2021-05-17T20:51:49Z")

</div>

Unfortunately not. When I try this it says I can’t find the channels:  
Collecting package metadata (current\_repodata.json): done  
Solving environment: failed with initial frozen solve. Retrying with flexible solve.  
Collecting package metadata (repodata.json): done  
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

- git-annex

Current channels:

- [conda.anaconda.org/conda-forge/osx-64](http://conda.anaconda.org/conda-forge/osx-64)

- conda.anaconda. org/ /conda-forge/noarch

- repo.anaconda. com/pkgs/main/osx-64

- repo.anaconda. com/pkgs/main/noarch

- repo.anaconda. com/pkgs/r/osx-64

- repo.anaconda. com/pkgs/r/noarch  
(I put a space between .com/.org because it wouldn’t let me post so many links)  
To search for alternate channels that may provide the conda package you’re  
looking for, navigate to

and use the search bar at the top of the page.

I tried some of the other channels like osx64 and noarch, but that didn’t work either.

When I type in conda config --show channels it says “defaults”

Any thoughts on that? What additional information would you need to test it?

---

<div class="post-metadata">

**Author:** ![fomightez](https://yyz1.discourse-cdn.com/flex031/user_avatar/discourse.jupyter.org/fomightez/32/495_2.png) [@fomightez](https://discourse.jupyter.org/u/fomightez)\
**Post date:** [May 17, 2021, 10:14pm UTC](https://discourse.jupyter.org/t/datalad-api-works-in-terminal-ipython-but-not-jupyter-lab-how-to-fix/9218/6 "2021-05-17T22:14:31Z")

</div>

I forgot that you are trying it on a Mac. That `%conda install` command I suggested above won’t work for you then because on the page I linked to there isn’t a MacOS icon listed [there](https://anaconda.org/conda-forge/git-annex) under ‘Installers’, like it does [here](https://anaconda.org/conda-forge/git-annex).

I cannot test what you need anyway because of the mac issue anyway. I’d be testing on a linux system.

You somehow need to point the environment running the notebook at where you installed git-annex on your computer. Usually a symbolic link is enough.  
From what you have posted about stuff run in your notebooks, it looks like it is looking for stuff in `~/miniconda3/envs/test/lib/python3.7/site-packages/datalad/`, but I suspect that was installed by pip. I don’t know though where the conda stuff would be placed in your file hierarchy. If you can determine where conda install stuff gets installed, you’d want to add to there the symbolic link from your local gut-annex install that you say works in the terminal. (I wonder for debugging if you need to think more general here and look for examples of ‘how to link Homebrew installations to Jupyter installs on a Mac’ or something like that, assuming you used Homebrew to install git-annex onto your Mac.)

If you have directions that said you could do this in a Jupyter notebook on a Mac that would be helpful to post. The Datalad documentation I looked at all talked about it being a command line program.