Jupyterpost: post from jupyterhub to a chat server (mattermost)

JupyterHub
1

Hi all,

The tool

I’ve freshly implemented a simple JupyterHub service jupyterpost (pypi, repo) that converts this code

from jupyterpost import post
from matplotlib import pyplot
pyplot.plot([0, 1, 0])
post(
    "Hello from Jupyterhub :)",  # Your message
    "@anton-akhmerov",  # The channel, can be @username
    pyplot.gcf(),  # Either matplotlib figure or bytes of the png image
)

into this chat message:
image

Right now jupyterpost only supports mattermost chat, but generalizing to more services (say slack) is within the project scope.

Implementation notes

I found that adding a JupyterHub service requires changing a lot of moving parts:

  • The service needs to be added to the hub
  • Proper credentials need to be passed to the service downstream
  • Users and in my case user servers need to get a permission to access it
  • User servers need to know where to find this service

To simplify adding the service I implemented a convenience function for modifying the hub config in place, so that all of the above points are achieved by this code after all the other configuration is done:

from jupyterpost import configure_jupyterhub

configure_jupyterhub(
    c,
    mattermost_token="your mattermost token",
    mattermost_url="https://your.mattermost.server/api/v4/",
    mattermost_team="your mattermost team name",
    jupyterpost_url="https://services.your.jupyterhub/services/jupyterpost",
)

Questions

  • Unfortunately I didn’t figure out how to compute the service URL (including protocol and domain) from the instance of hub config, that’s why I provide an additional config parameter. Any advice there?
  • Is there any practical advice for testing hub services?
3 Likes
2

Hi @Anton_Akhmerov,
that is a cool project!
Here are some ideas where this could go further:

  • How about not only posting the output to mattermost, but also the code?
  • Also, wouldn’t it be nice to skip the step of byte conversion with pyplot.gcf() ?
    Then one could stream a cell output and its generating code like it’s normally used without any modification.

A way that could be achieved would be by using cellmagic in the first line, e.g.

%%cellstream --message "Hello from Jupyterhub :)" --channel "@anton-akhmerov" 
import matplotlib.pyplot as plt
plt.plot([1,2], [10,20], c = "orange")

I’ve already implemented a very similar cell magic in the package jupyter_capture_output.
It can capture code, as well as text, image and video outputs.

1 Like
3

In general you can’t tell what the external facing URL for JupyterHub server is from the config file. For example, JupyterHub might run on an internal server http://192.168.0.1:8000 but be accessed via a HTTP proxy with a public domain name (or multiple names), and HTTPS.

However, it sounds like you’re connecting to the service from the singleuser server, which means you can use the same address that the singleuser server uses to communicate with JupyterHub. Start a terminal in your singleuser server and run

env | grep JUPYTERHUB_

You should see several environment variables such as JUPYTERHUB_API which is URL for the hub api, so you can easily substitute in your service path.

Depending on what you want to test you could do one or more of

1 Like
4

Excellent suggestion, thanks! I’ve implemented it now, and it does appear much more convenient. Now it looks like this:

compute_stuff()
%post @myself computation finished!

%%post coding-questions why does this code raise an error? -i

def f():
    locals()["foo"] = True
    return foo

f()

%%post project_channel check out my latest results

pyplot.imshow(data)
1 Like
5

%post @myself computation finished!

This notification looks very useful when a very long computation is done, thanks for this idea :slight_smile:

Here are some more thoughts regarding this project:

Processing the cell

How about wrapping the message in strings?

e.g.

compute_stuff()
%post @myself "computation finished!"

Cell magic can also take variables, so something like this would also be possible:

compute_stuff()
message = "computation finished!"
%post @myself $message

But then, the last line would also be posted.
This could be avoided by having a parameter NOT in the comments, that mutes certain lines from the stream (implementation can be done like here)

compute_stuff()
message = "computation finished!" # NOT
%post @myself $message

Posting to other platforms

It would be interesting to investigate if this posting option can also be possible from a JupyterLab notebook without using JupyterHub.

There are also many other platforms where auto-posting a Jupyter cell and its output can be useful, e.g.

discourse.jupyter, Twitter, Mastodon, Discord, Telegram and gitter.

Do you think that your project would be a good place for support of some of these platforms?
Or would it be more convenient to make separate repos for each platform?

Also, I saw that the project is currently on GitLab.
Therefore, the question: would you consider migrating the project to GitHub?
Advantages would be: it’s easier to contribute for people, and contributions will show up on the GitHub profile, people can star the project, one can use the GitHub CI/CD (e.g. for auto-publishing on pypi), and it’s possible to reference issues and prs in other GitHub projects.

6

This would make sense if the usage was confusing or error-prone otherwise, but so far I have no evidence for that.

That loses any advantage over a much more standard post("@myself", message). I think it makes sense to avoid implementing custom solutions if standard ones do the job equally well.

Yup, the bottom of the readme says that I consider it a good direction to go. Twitter isn’t going to be one of those because they are closing their API :man_shrugging:

The main application of using JupyterHub as an intermediary is avoiding credential management if both the hub and the message receiving service have the same users. I believe that the effort of configuring automated posting from an own notebook server is seldom going to be justified when one can copypaste manually anyway.

It’s not just gitlab, it’s my own gitlab :slight_smile: I really don’t care about github profiles, stars, or CI/CD (gitlab has its own, and so does mine). I recognize that it’s easier to contribute where one already has an account, but it’s a personal toy project. If it suddenly becomes popular, I’m happy to move it to github. Still, for the ease of code access here’s a github mirror (with the issue tracker turned off): GitHub - quantum-tinkerer/jupyterpost: Mirror of https://gitlab.kwant-project.org/qt/jupyterpost/

2 Likes