IPython process subsitution and "!"

General
1

Not sure if this is the correct forum, I have just been confused about this usage in a JupyterLab notebook.

Commands prefixed with “!” don’t seem to work for process substitution:

!wc -l <(pwd)
/bin/sh: 1: Syntax error: “(” unexpected

I assumed this was because “!” for some reason defaulted to using /bin/sh, not bin/bash (though bash is what is in my %env SHELL). On Ubuntu sh points to dash, so I tried just using a symlink to have it point to bash instead, but that just gives a similar set of errors (thought it does seem to change the shell):

!wc -l <(pwd)
[symlinkdir]/sh: -c: line 0: syntax error near unexpected token ‘(’
[symlinkdir]/sh: -c: line 0: ‘wc -l <(pwd)’

However, they can be done using:

!bash -c "wc -l <(pwd)"
1 /dev/fd/63

I am interested in why this is the case, but after a few days worth of searching I can’t work it out. Is this a fundamental limitation with how IPython runs “!”-prefixed commands (generally, or in JupyterLab)? Or is there a simple configuration that needs to be changed?

2

! maps to get_ipython().system which started as a wrapper for os.system, which unconditionally calls /bin/sh. IPython actually has control of this now, and in fact terminal IPython should already do what you want. This PR makes the kernel version (used in notebook) behave the same as terminal IPython.

1 Like
3

In the meantime I believe you should be able to use !! instead of a single !.

4

Thanks for the insight! That PR seems like a useful additional to the kernel.

!! does indeed work, as apparently does assigning the !-prefixed command to a variable (x=!wc -l <(pwd) versus simply !wc -l <(pwd)), for I assume related reasons.