How to get output model for a given cell in a JupyterLab extension?

stefaneidelloth · October 21, 2021, 12:21pm

I am able to access a cell model and its corresponding widget.
How can I get the corresponding output model? I would like to know the output expression (“b”) and its corresponding value (“3”). (My aim is to use this information for building a DAG of the “simple” cells, having a single named output.)

As a workaround, I could get the output value “3” from the div element of the widget dom e.g. with

let div = cellWidget.outputArea.node.children[0]
extractOutputValueFromDiv(div)

However, there should be a more elegant way, I guess?

For example, I would expect cellModel.outputs to include the output information I am looking for, but it seems to be empty (?).

=> What is the recommended way to access the output for a given cell model?

Related:

https://github.com/jupyter/notebook/issues/1175

https://discourse.jupyter.org/t/dag-based-notebooks/11173

https://stackoverflow.com/questions/69657393/how-to-use-events-in-jupyterlab-extensions/69658573#69658573

Part of the code I have so far:

function __observeNotebook(app){

	
	let notebook = __tryToGetNotebook(app);
	if(notebook){
		let model = notebook.model;
		let cellModels = model.cells
		cellModels.changed.connect(__cellsChanged, this);	

		for(let cellIndex=0; cellIndex < cellModels.length; cellIndex++){
			let cellModel = cellModels.get(cellIndex);			
			__observeCell(cellModel, notebook);
		}
	}		 

}

function __observeCell(cellModel, notebook){   
    cellModel.contentChanged.connect(cellModel => __cellContentChanged(cellModel, notebook), this);   
    cellModel.stateChanged.connect(__cellStateChanged, this);   
}

function __cellsChanged(cellModels, change){
    console.log("Cells changed:")
    console.log("type: " + change.type);
    console.log("oldIndex: " + change.oldIndex);
    console.log("newIndex: " + change.newIndex);
    console.log("oldValues: " + change.oldValues); 
    console.log("newValues: " + change.newValues); 

    if(change.type == "add"){
    	var newCellModel = cellModels.get(change.newIndex);
        __observeCell(newCellModel);
    }
}

function __cellContentChanged(cellModel, notebook){	
    let id = cellModel.id
    console.log("Content of cell " + id + " changed");

    let currentText =  cellModel.value.text;
    console.log(currentText);
   
    let cellWidget = notebook.widgets.find(widget=>{
    	return widget.model.id == id;
    });

    let outputArea = cellWidget.outputArea;
    let children = outputArea.node.children;
    if(children.length==1){
    	let output = children[0];
    	console.log(output);
    }
   
}

function __cellStateChanged(cellModel, change){
	let currentText =  cellModel.value.text;
    console.log("State of cell " + cellModel.id + " changed:");
    console.log("name: " + change.name);
    console.log("old value: " + change.oldValue);
    console.log("new value: " + change.newValue);
}

function __tryToGetNotebookCell(app){   
	var notebook = __tryToGetNotebook(app);
	return notebook
		?notebook.activeCell
		:null;    	
}

function __tryToGetNotebook(app){
	var notebookPanel = __getFirstVisibleNotebookPanel(app);
    return notebookPanel
        ?notebookPanel.content
        :null;
}


function __getFirstVisibleNotebookPanel(app){
	var mainWidgets = app.shell.widgets('main');
	var widget = mainWidgets.next();
	while(widget){
		var type = widget.sessionContext.type;
		if(type == 'notebook'){  //other wigets might be of type DocumentWidget
			if (widget.isVisible){
				return widget;
			}
		}
		widget = mainWidgets.next();
	}
	return null;
}

adpatter · October 21, 2021, 1:20pm

As I remember there is an array of outputs in the outputArea. Perhaps you’ll find it there.

(NotebookPanel#content.widgets[0] as CodeCell).outputArea.model

stefaneidelloth · October 21, 2021, 1:27pm

a) So far, I only found outputArea.node.children, as already shown above.

b) The outputArea.model.list seem to be empty.

c) Same for cellWidget._output.model.list

d) outputArea.model.contentFactory.createOutputModel()

might be a way to go, but I don’t know what I should pass as argument.

baggiponte · October 24, 2021, 8:59am

I don’t really know if this can be of any help, but people at Julia have developed Pluto.jl, interactive notebooks for Julia. Maybe they had to solve this problem!

krassowski · October 24, 2021, 11:56am

To start with, I am not convinced that trying to obtain this information from JupyterLab frontend is the prime way to approach the problem, but let’s entertain it for educational reasons.

First, you can get the output of the cell from its model and your attempts to use model.list were not too far off. You may have seen empty values because the operations are asynchronous, i.e. the first time when Output Area becomes visible it is not yet populated. This can be resolved by listening on changes to its content:

outputs-demo

Code:

{
  id: 'CellOutputExample',
  autoStart: true,
  requires: ["@jupyterlab/notebook:INotebookTracker"],
  activate: function(app, notebookTracker) {
      notebookTracker.widgetAdded.connect((tracker, panel) => {
          let notebook = panel.content;
          const notebookModel = notebook.model;
          notebookModel.cells.changed.connect((_, change) => {
              if (change.type != 'add') {
                  return;
              }
              for (const cellModel of change.newValues) {
                  // ensure we have CodeCellModel
                  if (cellModel.type != 'code') {
                      return;
                  }
                  // IOutputAreaModel
                  let outputs = cellModel.outputs;
                  if (!outputs) {
                      continue;
                  }
                  outputs.changed.connect(() => {
                      console.log('Outputs of the cell', cellModel.id, 'in', notebook.title.label, 'changed:');
                      console.log(
                          '\tThere are now', outputs.length, 'outputs:'
                      );
                      for (let i = 0; i < outputs.length; i++) {
                          // IOutputModel
                          const outputModel = outputs.get(i);
                          console.log('\t\t', outputModel.data);
                          // also has `outputModel.executionCount` and `outputModel.metadata`
                      }
                  });
              }
          });
      })
  }
}

(Note this is JavaScript for GitHub - jupyterlab/jupyterlab-plugin-playground: A dynamic extension loader for JupyterLab, not a proper TypeScript code, it’s Sunday, I’m not opening my IDE today).

Second, I would really avoid observing the results of execution by engaging the DOM or widgets at all; instead I would listen to the NotebookActions.executed which also enables you to handle errors in a clean way. I also highly recommend sticking to the public members (do not use underscored properties like ._output). Peeking with the debugger/console is a good approach, but it should be secondary to reading the API reference which let me quickly create the example above. I usually go back and forth between logging to console, API reference and - if needed - the actual source code of JupyterLab (and extensions). Also, a well-configured autocompletion can help a lot!

Third, there is a lot of prior work mentioned in the other DAG thread but one other extension not mentioned there, but probably very relevant to what you are trying to achieve (though this might not be immediately obvious) is GitHub - nbsafety-project/nbsafety: Fearless interactivity for Jupyter notebooks. as it tracks the execution of the cells, their results and modifies the display of the notebook on the frontend to guide users on out-of-order execution. I think that building upon their approach is one of the most promising ways to implement DAG notebooks (with equally GitHub - davidbrochart/akernel: Asynchronous, reactive Python Jupyter kernel. being another promising projects).

Finally, in another thread you hypothesise that getting the result of cell execution is difficult in JupyterLab and Maybe that is one of the reasons why other implementations of observable notebooks are based on extra kernels?. While I hope I demonstrated above that the premise is not entirely correct, I want to also point you to one of the previous experiments, GitHub - cphyc/ipyspaghetti which implements what I believe is the most functional DAG notebook for JupyterLab 3.x and it does not require an extra kernel. Maybe, just maybe, contributing to that project would be a better way forward rather than trying to re-implement it from scratch?

stefaneidelloth · October 25, 2021, 2:34pm

Many thanks for the detailed answer!

To First:
Juhu, With your help I am now able to access the output values:

use outputs.get(index) method inside index loops instead of outputs[index] or iterations
be careful about timing

In order to find out what expression caused some output, I have to evaluate the last line
of the cell input, because the output model does not include that information (please correct if this is wrong).

To Second:

I also highly recommend sticking to the public members (do not use underscored properties like ._output )

Fully agree.

reading the API reference

Once I know where to look/what key terms to search for, the api doc might be helpful.
Where can I find the doc for the “outputs” property of the cell model,
saying that I should use outputs.get(index) instead of outputs[index] and that outputs is not iterable?

That info does not seem to be included in

https://jupyterlab.readthedocs.io/en/stable/api/classes/cells.codecell-1.html
or
https://jupyterlab.readthedocs.io/en/stable/api/interfaces/cells.icodecellmodel.html
or the search:

=> This forum is a very important/helpful supplement to the api doc.

I updated a related answer on SO to clarify where to find documentation. Please extend/correct if you want:

To Third and Finally

I included nbsafety as another option in an overview here:

Maybe, just maybe, contributing to that project would be a better way forward rather
than trying to re-implement it from scratch?

Yes, I would prefer to build on existing solutions. I am still in the orientation phase, trying to follow multiple paths (including options like observablehq and starboard) and find out what might give the best match.

Indeed, I have several ideas that I would like to combine for reactive data flow models:

krassowski · November 7, 2021, 12:12pm

https://jupyterlab.readthedocs.io/en/stable/api/interfaces/cells.icodecellmodel.html#outputs does include the description of outputs property:

The part after colon (IOutputAreaModel) is a link to the description of the type of the outputs - after clicking on it you can see that it provides get(index: number): IOutputModel along with many other methods (like .set()) and properties (like .length). It would be more intuitive to use if it was iterable indeed - I can imagine that this would be a welcome contribution if you want to make it so (but please consult with more experience contributors than me first by opening an issue in JupyterLab repo first - maybe there is a reason for not having the iterator implemented here - though I think it is unlikely)

Topic		Replies	Views
Output Callback in JupyterLab Extension Widgets jupyterlab , help-wanted	0	362	November 4, 2023
What is a `model` in JupyterLab? JupyterLab	3	916	April 13, 2021
How to get a reference to a CodeCell? JupyterLab	2	859	March 27, 2021
DAG Based notebooks? JupyterLab	7	4891	May 11, 2022
Accessing cell subelements using API JupyterLab	2	754	August 10, 2020

How to get output model for a given cell in a JupyterLab extension?

Related Topics