Where do I need to copy a config.json or overrides.json to so that mybinder accepts it?

Binder
1

My task seems simple: I am using jupyter-ai in a jupyter lab / mybinder.org environment and my goal is to configure it in a way that there is an LLM already pre-selected and -configured when the jupyter environment starts.

After a lot of trial and error I have now managed to do this successfully on my local linux system. However, I am not sure if I achieved this by creating an overrides.json which lives in the .local/jupyter folder, or by simply starting jupyter lab once with jupyter lab --AiExtension.default_language_model=openai-chat:gpt-4. There is now a config.json in a jupyter-ai subfolder, which includes the setting for the default LLM and which jupyter lab automatically seems to use when it starts (I assume either the command created it or the overrides.json). So the default LLM in jupyter ai is now set to openai locally every time I start jupyter lab (just with ā€˜jupyter labā€™), which is - technically - exactly what I want.

However, I also want this to work when I start a binder via mybinder.org from my github-repository and I still absolutely fail to do that. I have tried postBuilds, start files, config.jsons and overrides.jsons, I had mybinder move those files to various locations at start-up - none of it worked. Through the binder logs, I know that the postBuild file executes alright and the folders I need are created resp. the config or overrides file moved or copied to the right places as well, but itā€™s still not working. Either mybinder is building the container successfully but then never starting the server, or it starts eventually but then jupyter-ai isnā€™t even there. My latest approach now has is me using a start file with the following content:

#!/bin/bash
# Start JupyterLab with a custom configuration
exec jupyter lab --AiExtension.default_language_model=openai-chat:gpt-4

and a postBuild looking like this:

#!/bin/bash
# Make the start script executable
chmod +x binder/start

which at least isnā€™t messing everything up - the server starts and jupyter-ai shows up and is available. BUT: It still has done nothing to pre-select an LLM.

I am sure itā€™s probably actually really simple once you know which command or conig-file needs to go exactly where, but I feel like Iā€™ve tried every possibility and itā€™s just not working. Can anyone help??

2

In 2022 it was jupyter_config.json in the root of the repo, see here. (I wonder if you could also put it in the binder directory if you have one? I havenā€™t tested that or any of this recently.)

1 Like
3

Dear @ssp24,

thanks for the question.

You can customise the Jupyter Lab settings with a file named overrides.json as described in JupyterLab on Binder ā€” JupyterLab 4.2.4 documentation.

You can reproduce the mybinder.org build using jupyter-repo2docker ā€” repo2docker 2024.07.0+9.gafaa6e3 documentation. Installation is like any other Python package:

python3 -m pip install jupyter-repo2docker

And you can build the current working directory with

jupyter-repo2docker .
2 Likes
4

Thank you, I tried this but it didnā€™t do anything, unfortunately. But might also be that my config-file isnā€™t entirely correct, it currently looks like this:

{
  "AiExtension": {
    "default_language_model": "openai-chat:gpt-4"
  }
}
5

Thank you, Iā€™ve tried the overrides.json apprach again as well but it didnā€™t work either. I can tell that it something, though, as it made the jupyter-ai widget disappear completely instead of setting a default LLM, so maybe this is one step into the right directionā€¦

My overrides.json looked like this:

{
  "AiExtension": {
    "default_language_model": "openai-chat:gpt-4"
  }
}

and my postBuild like this:

#!/usr/bin/env bash
set -eux

# Create the directory
mkdir -p ${NB_PYTHON_PREFIX}/share/jupyter/lab/settings
if [[ $? -eq 0 ]]; then
  echo "Successfully created the directory ${NB_PYTHON_PREFIX}/share/jupyter/lab/settings"
else
  echo "Failed to create the directory ${NB_PYTHON_PREFIX}/share/jupyter/lab/settings" >&2
  exit 1
fi

# Copy the overrides.json file
cp overrides.json ${NB_PYTHON_PREFIX}/share/jupyter/lab/settings
if [[ $? -eq 0 ]]; then
  echo "Successfully copied overrides.json to ${NB_PYTHON_PREFIX}/share/jupyter/lab/settings"
else
  echo "Failed to copy overrides.json to ${NB_PYTHON_PREFIX}/share/jupyter/lab/settings" >&2
  exit 1
fi

I had them both in a binder-folder and added the echo messages to see if the postBuild was correctly addressed and able to do itā€™s job and this part seems to work alright (the log messages say that the folder was created successfully and the file copied as well). However, after copying the overrides.json into the settings folder the jupyter-ai extension disappeared completely from my mybinder-image.
I also tried copying the file to this folder: mkdir -p ${HOME}/.jupyter/lab/user-settings/@jupyter-ai/ai-extension, which I found as an alternative, but the result was exactly the same: instead of setting a default LLM, jupyter-ai disappeared entirely.
Any idea what I am doing wrong here?

I still have to try and reproduce the mybinder-build with repo2docker as suggested, but not sure how quickly Iā€™ll be able to do that. I havenā€™t quite wrapped my head around repo2docker yet, but I will try this as well.

If anyone has any other ideas, Iā€™d be more than happy to try them, I just canā€™t imagine that this is something that canā€™t be solved.

1 Like
6

So I had a look at what happens with my local installation and it seems that starting jupyter lab once with the long option --AiExtension.default_language_model=openai-chat:gpt-4 seems to create a config.json in my home-folder under .local/share/jupyter/jupyter_ai, which looks like this:

{
    "model_provider_id": "openai-chat:gpt-4",
    "embeddings_provider_id": null,
    "send_with_shift_enter": false,
    "fields": {
        "bedrock:ai21.j2-mid-v1": {
            "model_kwargs": {
                "maxTokens": 200
            }
        }
    },
    "api_keys": {
        "OPENAI_API_KEY": "test-key-will-not-work"
    },
    "completions_model_provider_id": null,
    "completions_fields": {}
}

So I tried to my mimic this and while I havenā€™t succeeded so far in simply including the long option somehwere in my binder-build, I have created the config.json and used the postBuild file to create a folder .local/share/jupyter/jupyter_ai and copy the config.json into it. Using the terminal in my mybiner build, I can see that this works, the folder is created and the config.json in place, but: Once again jupyter-ai has completely dissapeared from the build, thereā€™s no widget at all.

I also tried using a start-file instead but this keeps the server from starting entirely.

Using a Dockerfile letā€™s the server start properly but then jupyter-ai disappears again entirely. This is slowly driving me insaneā€¦

1 Like
7

In case someone else stumbles on that problem: after several attempts trying to achieve something similar, what worked for me was to create a file etc/jupyter/jupyter_jupyter_ai_config.json under my conda prefix, with the following content:

{
  "AiExtension": {
    "default_language_model": "aristote-provider:casperhansen/llama-3-70b-instruct-awq"
  }
}
2 Likes
8

Thanks very much for the reply and glad this works for you! Could you maybe elaborate a little bit further on where exactly you placed the file? I am not sure what to make of ā€œunder my conda prefixā€ in a github/mybinder.org-setting, or do you refer to a local installation?

1 Like
9

Hi @ssp24!

My contexts are:

  • local usage on personal computer, or
  • docker images used locally or on our virtual environment infrastructure (mydocker).

In both cases, this is using conda environments. So for example the full file name is `/opt/conda/etc/jupyter/jupyter_jupyter_ai_config.json if installed in the base environment of one of the docker images from the jupyter docker stack. Presumably this should be the same in binder if configuring the environment through a conda environment.yml or pip requirements.py since IIRC they use one of these docker images.

Hope that helps!

1 Like
10

Thank you! Seems like I really need to look into the docker image option, but I havenā€™t quite gotten my head around that yetā€¦