Ipython - Function argument

JupyterHub
1

Hello,
(that’s my first post and I hopw I am at the right place).
I installed Jupyter and use the notebook for preparing a personal project for manipulating Sentinel S5P products.
I am writing an ipython program that calls a program (underlying shell): harpmerge. The program is called with “!harpmerge”
harpmerge is called with several arguments and the last two ones are: “input filenames” and and “output filename”. I can type in text those 2 argument without quote:
!harpmerge … /home/user/data/inputFileNames.nc /home/user/data/output.nc*
I want to replace the hard-coded text with variables such as:
inputFiles = "/home/user/data/inputFileNames.nc"*
outputFile="/home/user/data/output.nc"
!harpmerge … inputFiles outputFile
The output I get is that the inputFiles directory does not exist.

I changed to
inputFiles = "/home/user/data/inputFileNames.nc"*
outputFile="/home/user/data/output.nc"
!harpmerge … $inputFiles $outputFile
It still does not work.

Would someone know how to pass the value to harpmerge?

2

Have you tried curly brackets like in this example? IPython and Shell Commands | Python Data Science Handbook

3

Thanks for the hint.
I tried curly brackets, &… without success

4

First, trying to decipher your asterisks and italics from real code is a problem. Discourse allows you to use markdown in posts. Please use a code block begun with triple ticks and then closed with another set to paste in code, see here and look for what it shows below ‘Another (short) example, to demonstrate how to wrap your code blocks’. If it lets you edit your post, you can fix it. I cannot tell if that asterisk is real or from your placing in italics?

Second, are you sure you are pointing at a directory since you are seeing inputFiles directory does not exist.? The man page for harpmerge says it will take a directory but your examples don’t look like directories.

Assuming the asterisk is extra and you are pointing correctly…
One thing to try would be to run in a cell or in IPython console:

inputFiles = r"/home/user/data/inputFileNames.nc"
outputFile= r"/home/user/data/output.nc"
!harpmerge … {inputFiles} {outputFile}

If that asterisk is important, then you want it inside the variable. Not inputFiles = "/home/user/data/inputFileNames .nc"* (Or follow the advice at Harpconvert and harpmerge anaconda powershell - #2 by sander.niemeijer - HARP - Atmospheric Toolbox ?)
So try in a cell or in IPython console (try this set with and without the r in front of the fist quote):

inputFiles = r"/home/user/data/output.nc*"
outputFile = r"/home/user/data/output.nc"
!harpmerge … {inputFiles} {outputFile}

If that doesn’t work, fall back and check fundamentals…
Do shell stuff without harpmerge involvement first, such as specify something to pass to echo with curly brackets define in Python earlier to convince yourself that Jupyter/IPython and shell commands are fine. Most likely is you aren’t pointing at the inputFiles directory correctly since you are getting that error. Did you try pointing at the directory and just running ls {inputFiles} to list the contents? Did you try moving things to where you are running so the path is simpler?

5

Thanks for your answer.

The file path is correct and if I use “ls” with that path I get my files displayed.
The “*” is also correctly placed. I can remove the last letters anyway (it does not change anything).
The used path is simple and located just two levels below my home directory ("~/Documents/Data"). By the way, I also tried the code using the relative path: ~/Documents/Data, without more success.

I modified the code as you suggested:
input_files = r"/home/myusername/Documents/Data/S5P_OFFL_L2__NO2____*"
output_file = r"/home/myusername/Documents/Data/converted_NO2_2020_final.nc"
print ("input files: ",input_files)
print(“Start harpmerge:”)
!harpmerge -ap ‘bin(); squash(time, (latitude,longitude))’ \
-a ‘latitude >= 40 [degree_north]; latitude <=60 [degree_north]; \
longitude>=-22 [degree_east];longitude<=26 [degree_east]; \
tropospheric_NO2_column_number_density_validity>75; \
bin_spatial(360,40,0.05,960,-22,0.05); \
derive(longitude {longitude});derive(latitude {latitude}); \
keep(latitude,longitude,tropospheric_NO2_column_number_density,weight)’ \
-l \
{input_files} \
{output_file}

The output is:
input files: /home/yvesdoat/Documents/Data/S5P_OFFL_L2__NO2____*
Start harpmerge:
ERROR: could not find ‘{input_files}’

6

I think you’ve ruled out this being a Jupyter/IPython issue. You are having issues with harpmerge using the files you want as input on your machine. Have you tried to either use bash or ipython to cycle through your directory containing the input files to make a file as described here and then design your command like suggested there?

If that doesn’t work, next I would suggest asking a the Atmospheric Toolbox discourse forum. You can point to here to indicate what you you have tried. (Before posting there, I would still encourage you to learn how to post code blocks to discourse. You just need to flank them with triple back ticks as shown in the documentation I pointed out previously.)

1 Like