External API calls to trigger JupyterLab features

Hi,

I am currently working on integrating JupyterLab to our internal software ecosystem and I have a requirement where I need to use an external API call, made by one of our internal services to open a notebook within JupyterLab and populate it with specific content. Does anyone know if this kind of feature is supported? i.e. the UI interaction is outside the realm of the JupyterLab UI. I have looked into some online material on using event handlers for this purpose, but I still don’t know how to wire this up. Any help would be greatly appreciated!

Thank you

3 Likes

+1. I am trying to do something similar by opening a dynamically created ipynb file as a new tab in the JupyterLab UI. Is there’s a way to do this through terminal, backend code, or any other way?

Any help is appreciated! Thank you!

1 Like

You can create a JupyterLab extension that will listen to commands via any protocol you would like - you are in control of what your extension listens too and via extension you can control the UI fully (preferably by executing commands, but you could also directly interface with the DOM).

1 Like

Does this mean,

  • “something happens at some time on someone else’s computer, and it wants to make something happen in my JupyterLab”
  • “something happens in my JupyterLab, and JS in my JupyterLab makes an external API call and then makes something happen in my JupyterLab”

If the former: yeah, there isn’t a way on vanilla jupyterlab of doing that. The closest would be:

  • you have JS running in the browser
    • jupyterlab client is running in an iframe, served from the same domain
    • the jupyterlab server process was launched with --expose-app-in-browser

Then your outer code would be able to use iframe.contentWindow.jupyterapp.commands.execute(commandId, {arg: value}) to do things.

This is kind of a hack.

There might be appropriate commands to just do what you need, but usually stringing them together can be enough. As to what commands are available… well, the best way to find out is to use that setting, and check it out in the browser console. Otherwise, they can be found by inspection. though extensions often add their own.

All that being said: a custom labextension is going to be a far more documented path. Once you get all your CORS squared away, it might boil down to one non-boilerplate function… you’ll have access to window.fetch, which can pretty much do whatever you need, the JupyterLab app itself, and anything else you care to bring along… but again, very possibly, all you’ll need is app.commands.

An even more exotic route, if you actually want to make persistent notebooks backed by your API is the IDrive API, as implemented, for example, by jupyterlab-github

1 Like

@bollwyvl Thanks a lot for your response! First of all, apologies if I make any errors in my comment, I am not a front-end developer so this is quite new for me. My requirement closely aligns with the former bullet point in your comment, i.e.

“something happens at some time on someone else’s computer, and it wants to make something happen in my JupyterLab”

Currently, our Jupyterlab setup is embedded within an iFrame in the browser, and all that iFrame is doing right now is just redirecting users to “/jhub” whenever they click a “Launch Jupyterlab” button. Now I have a custom extension within Jupyterlab that exposes a couple of buttons that execute some custom commands in the Typescript code. This is of course, within the realm of Jupyterlab. I just want to call the same functionality, but when a button is clicked on our UI.

From what you said, it seems like executing the command directly within the iFrame seems like a viable option. Could you point me to any available documentation for this please? This would really help.

To your second point on developing a custom labextension, this seems like the better approach to follow, however I am not sure if I completely understand what you mean. My trigger is actually coming from an external source, i.e. a button click outside of Jupyterlab; or an external API call made to the Jupyterlab client-side extension somehow. I’d like to trigger a Jupyterlab command when this happens. What would trigger the windows.fetch in this case?

I’ll take a look at the iDrive API as well, and see if it could be useful for my use-case.

@krassowski Thanks a lot for your suggestion! Could you please point to any examples you know of that do this? I would like to directly modify the DOM but currently I only know how to make API calls to a server side extension, but don’t actually now how to modify Jupyterlab functionality by calling external Javascript code.

The --expose-app-in-browser thing is, as I said, a hack, and probably not something you should rely on, and as such isn’t rely documented as a sound integration technique. But as it can be enabled without re-building anything, it can be quick to explore in the browser console, which is going to be more instructive than narrative.

The labextension technique, on the other hand, has extensive documentation and automated tools.

If you’re already building a labextension, you can add your own iframe API with postMessage. Note, too: this kinda smells like a hack. That’s why this kind of capability isn’t in Lab in the first place… and probably won’t be added, but might make sense as a dedicated extension, especially if it allowed for mapping arbitrary postMessage payloads to JupyterLab commands, and handled some of the security junk.

Anyhow: roughly around here, this untested/unchecked code is kinda what one might need:

const activate(lab: JupyterFrontEnd) => {
  const MY_URL = 'https://my-app.com';
  const {commands} = app;
  
  const existingFeature(args) => /* ... */;

  const onMessage(event) => {
    const {source, origin, data} = event;
    if (origin != MY_URL) { return }
    existingFeature(JSON.parse(data));
  };

  window.addEventListener("message", onMessage);
};

Then, from outside:

const frame = document.querySelector('iframe#jupyterlab');
const button = document.querySelector('button#do-thing-in-lab')
button.addEventListener('click', function() {
  frame.postMessage(JSON.stringify({do: 'thing'}));
});
1 Like

I think the labextension route is the way to go, and this example will help me get a proof of concept going. I’ll try it out and update here with any issues. Once again, thanks a lot for your help @bollwyvl .

@bollwyvl - would this be the correct way of invoking jupyterlab with the expose browser option from the Jupyterhub config file?

c.Spawner.cmd = ['jupyter-labhub']
c.Spawner.args = ['--expose-app-in-browser']

I guess… i am not super familiar with the rest of your setup.

However, in a deployed setting, instead of args, you may want to use a config file (not everything you want to tweak has args) and just have an arg that loads that, e.g. a jupyter_config.json like:

{
  "LabApp": {
    "expose_app_in_browser": true
  }
}

though i don’t know where you’d put that…

So, in our case, we have a Jupyterhub setup which authenticates users based on our internal auth logic, and spawns a single-user Jupyterlab server for each authenticated user. Internally, it’s using the jupyter-labhub spawner, which I think comes from this code here. All the config options we’re providing right now flow through the jupyterhub_config.py which is the Jupyterhub’s config file. So I was thinking that maybe we’d have the option to do so within there itself. I could not find too much information about this specific scenario online.

I also tried what you mentioned earlier:

Then your outer code would be able to use iframe.contentWindow.jupyterapp.commands.execute(commandId, {arg: value}) to do things.

I could not find the jupyterapp property (most likely because the --expose-app-in-browser option is set to False). I did however find something called iframe.contentWindow._JUPYTERLAB which did not have any commands property.

iframe.contentWindow._JUPYTERLAB

well, _JUPYTERLAB may be accidental. The documented feature is to use the flag, and then rely on that… but again recall that this is really a defveloper-focused feature.

Keep in mind, though: unless it’s your desire to allow folk (and any extension they may install) to access everything in your parent application from JS, one should almost always run jupyter on a different domain, which will make this flag approach not work. The extension-implementing-an-iframe approach is going to be more robust and secure.

Yes, you’re right. Our desire is not to allow any user to access all the functionalities of the parent application via JS, as this can be insecure. I’m still in the process of implementing the custom extension and will see if that works. Would I have to set the --expose-app-in-browser flag for this method as well?

can be insecure

Welcome to remote-code-execution-as-a-service!

Basically, do everything you can to firewall a user’s jupyter server and kernels from your app, and assume that someone will try to abuse it. See the recent ChaosDB for examples of how this can get totally screwed up.

Would I have to set the --expose-app-in-browser flag for this method as well?

No, that’s the point of all of the MY_URL junk in the iframe skeleton above.

Thanks for sharing the ChaosDB blog, I had no idea. We have started taking some basic steps towards making our internal Jupyterlab secure, e.g. disabling terminal access, restricting functionality to admin users, etc, and we only hope to take it further!

By the way, the custom extension worked, and I was able to submit a postMessage and reactively run a command. So this paves a way for us to create a UI client library that acts as a wrapper to execute internal Jupyterlab commands! Thanks a ton for your help!

1 Like

Great, sounds good.

But of course, remember: untrusted code-execution-as-a-service can’t ever be made 100% safe for the machine it runs on and any machine it can connect to. Disabling terminal? Whatever, I’ll just use ! commands. Disabled those somehow? I’ll use any one of the 50 ways python gives me to run scripts. I’ll write a file out, and import it, and abuse the python loading mechanism.

See also How to control code executed in Jupyter Notebook (e.g. block malicious code execution) - #2 by bollwyvl

1 Like

Yes, such issues will always exist when using Python, as it’s so flexible in so many ways. For example, I could just use the eval method to execute anything I want, and that is an inherent vulnerability within Python. I guess moving to more containerized architectures might help localize the nature of the attack if there ever is one.