Better way to communicate from kernel from to jupyterlab extension than prints?

I am building a jupyterlab extension that executes python code in the kernel on the users behalf. Right now I am executing code very similarly to the extension tutorial where I send code to the kernel as a string then read the output when the result is printed.

Since I have many functions I want to execute I have put them in the same python module as my extension. So the strings I send to the kernel to execute look like

kernel.execute_code("mymodule.do_something()")

Then in python when the do_something() function is called it prints the result. like

def do_something():
    print(3)

My question is if there is a way for me to return the output and not have to read it in the javascript code as a printed output? For instance by just returning the result like an API endpoint instead of printing. Many of my returns end up being printed long serialized JSON strings from python which feels suboptimal.

Would comms help here, could anyone point me to some examples of how to do this with comms?

Would comms

One can do custom comms, but there’s already so much value from the widget bus, for which there are many, many examples. This is also likely to be more portable to other frontends.

One can entirely ignore the synced traits stuff, and just use custom messages to implement almost any message architecture, plus it supports things like binary buffers (numpy arrays, images, etc) which might be appropriate.

like an API

Indeed. Building a well-typed JSON schema for what goes back and forth can also help protect the user’s kernel from an extension doing “bad things,” and code (with our without validation) can be generated from such as schema. Or, the “reference” implementation can live in python or typescript, from which schema (and the missing client/server) can be generated.

update:

examples

Ignoring the request for custom comms, ipycanvas basically does canvas-commands-over-custom-messages for canvas commands:

Thanks for the reply!

Part of my difficultly is that my extension lives in the sidebar instead of inline like a normal widget. I have used traitlets before for inline widgets and am trying to figure out how to get that functionality when my widget is in the sidebar so the user never has to execute any code for it to show up.

For instance if I make a python class that extends widget like in the canvas example you linked like
class MyWidget(Widget): I’m not sure when this would ever be instantiated?

I have some screenshot in our github readme to show where I am putting the widget: GitHub - cmudig/AutoProfiler: Automatically profile dataframes in the Jupyter sidebar