JupyterAI very bad inline completions

Hey there,

I got the inline completions working with my OpenAI API Key.
I tried using multiple models (davinci, 3.5-turbo, 4-turbo) and they all produce really bad completions (usually way too long or what seems like copying from some API documentation).
When trying to use it inside parenthesis it doesn’t suggest anything at all.

I don’t know what’s needed to debug but I’ll provide whatever needed.

Example of completion -
Input code (what I typed then stopped typing) - px.li (I imported plotly.express as px)

Suggested completion -

px.line(data, label="data")
px.scatter(data, label="data")
px.box(data, label="data")
px.histogram(data, label="data")
px.box(data, label="data", color="color")
px.box(data, label="data", color="color", aspect_ratio=0.5)
px.box(data, label="data", color="color", aspect_ratio=0.5, color_range="color")
px.box(data, label="data", color="color", aspect_ratio=0.5, color_range="color", range_x=[0, 1])
px.box(data, label="data", color="color", aspect_ratio=0.5, color_range="color", range_x=[0, 1], range_y=[0, 1])

The above code produces a chart that looks like this:

import matplotlib.pyplot as plt

# Plotting 3 discrete distributions.
# Discrete distributions are performed by px.box.
# range_x=[0, 1] and range_y=[0, 1] allows for plotting the data
# within a range of [0, 1] on the x and y axis.
# color_range="color" and color="color" allow for color selection.
# aspect

When using GitHub copilot it always suggests something reasonable like a line plot with dummy data.

This is down to three things:

  • the model choice - while you tried some models, you are using models tuned for chat not for code completion
  • prompting - each model needs slightly different prompt to perform best for inline completion, but jupyter-ai uses the same prompt for all models by default; you can customize this if you know your way about extending jupyter-ai with a custom model provider, but unfortunately the prompt template is not exposed in the UI right now
  • context - the model often does not have enough context because it only sees the current cell; again custom providers can add full notebook content to the context

If you would like options for the above to be exposed in jupyter-ai, please open an issue on jupyter-ai repo for your voice to be heard. I think this should be the case, but I am not in a best position to convince devs from the company which maintains jupyter-ai that this is the way (saying this as a person who implemented the inline completer in both jupyterlab, jupyter-ai and elsewhere).

3 Likes

One more trick that some IDEs do is having a second pass filter on the suggestions to decide if those are good quality. This can be done using smaller models (and sometimes just with static analysis). I would love to see a simple version of that in jupyter-ai or an extension too.

1 Like

If you like the copilot suggestions more, you can use it directly though with GitHub - mbektas/notebook-intelligence: AI coding assistant for JupyterLab, powered by GitHub Copilot - it sends much more context (which will be slower for very long notebooks) and uses (unofficial) Copilot API.

1 Like

Thanks a lot!
The latter extension you mentioned seems like a good choice.
I am actually looking to contribute, do you think it would be a good idea to create a provider to be code completion oriented?
@krassowski

I think so. I do not know if that should initially be an extension, or part of jupyter-ai, but if it was good at providing inline suggestions I would use it and recommend it to others.

1 Like

@krassowski
I actually just tried the Notebook-intelligence extension you mentioned and it’s pretty good.
I’ll anyway try to take a look at making a new provider, if it’s not too complicated