Does JupyterLab language registry's addLanguage support LRLanguage as input?

JupyterLab
,
1

I am trying to add a syntax highlighter and after searching for how JupyterLab declares the default languages, I came across wrapping a LRLanguage from @lezer/generator in the JupyterLab source. Is this a supported way to add a new language support for extensions? I made a basic LRLanguage using the same format as in that file but if I create a fenced code block it doesn’t highlight my code. I am getting no errors in the inspector.

Can I get some help determining what is wrong? My code is posted here: algorithm_plugin.ts, highlight.ts.

1 Like
2

There are two code highlighters: for code and for markdown. What you did should get you code in a file or code cell highlighted, but for markdown it is a bit more complex (I assume that by “fenced code block” you mean using it in markdown - is this correct)?

1 Like
3

Yes, I am looking for markdown highlighting. I do not need file or code cell highlighting. My testing fenced code is simply:

``` algorithm
Hello
```
4

Could you check that the code block is parsed by your language (you can look at the Dom nodes structure for that)?

Normally that should work. What I think will not work is the highlights because you need to match your token with highlighted token:

If I’m correct, what you will need to highlight you token (aka your tag) is to provide a custom codemirror theme through using that lab plugin

2 Likes
5

Hello,
You then create a JupyterLab extension that registers your language with CodeMirror using specific MIME types and file extensions. Ensure your language identifier matches in fenced code blocks for proper integration, and monitor the browser console for any errors to debug effectively.
Thanks

6

Looking at the inspector, I can see the HTML and it seems to be appropriately as “algorithm” which is the name I’ve been using.

So assuming I am parsing my fenced code correctly, I should just need to provide a syntax highlighter scheme to CodeMirror which should then get applied to my parsed code?

7

Yes

Or easier, you use token tags that are already highlighted. It means you need to map your token to known tags; see Lezer Setup Example

A list of known tags is available there:
https://lezer.codemirror.net/docs/ref/#highlight.tags

Note: not all tags get highlighted in the default Jupyterlab codemirror theme. Hence the possible need for you to provide your own theme.

1 Like
8

I’ve modified my extension to add a custom theme to the theme registry, but it still isn’t working. I updating my Lezer highlighter to use tags seen in jupyterHighlightStyle so I can reuse it. I am seeing no errors or warnings on compilation or when I create fenced code in Markdown using my above template. When I create a .algorithm file for testing, I see that the Algorithm language is chosen, but I don’t see any highlighting.

Can you expand on what you mean about checking the DOM to ensure the language is getting properly recognized? I worry that while the language is recognized, the parser is failing somehow. To make testing easier I changed the grammar to be the Lezer example.

Here is my updated plugin and highlighter.

9

I forgot about it. But actually we have a test for that in lab that should help you:

The span elements with weird class names are applied by the highlighter.

That test uses a custom language defines in

Then we produce the JavaScript file for it:

In your case, I’m a bit worry about your syntax loading highlight.ts within a snippet.

1 Like
10

I’ll install jest, (figure out how to use it it,) and try to make the foo test case example you posted in my project.

In your case, I’m a bit worry about your syntax loading highlight.ts within a snippet.

What do you mean by this? Do you mean how I am adding the syntax highlighting in the .grammar given to Lezer?

11

What do you mean by this? Do you mean how I am adding the syntax highlighting in the .grammar given to Lezer?

To be included in JupyterLab, your JavaScript assets are bundled using webpack under the wood. The link within the CodeMirror lezer grammar is definitely ignored by webpack. So I’m not sure the code get properly packaged.

12

So using your example I was able to create a Markdown parser, and while it compiled correctly once, when I try to re-compile my project I get a type error complaining about how 2 of the same types aren’t actually the same. I am not terribly familiar with TypeScript but it seems it is because I am getting the LRParser from @jupyterlab/codemirror and @lezer/lr but I am not sure how I can convince Typescript they are the same.

Here is the error I have been getting.

13

Hey

assuming you use jlpm, you can solve this by adding an entry resolutions in your package.json:

resolutions: {

@lezer/lr: “^1.4.0”

},

14

I am using jlpm. I manually added the line you mentioned but am still getting the same error even when doing a clean build. Is there anything additional I need to do?

15

You can try running

jlpm dlx yarn-berry-deduplicate
jlpm

Could you share your package.json?

16

I ran those commands and now am seeing the following in addition to the previous errors:

error TS2688: Cannot find type definition file for 'cookie'.
  The file is in the program because:
    Entry point for implicit type library 'cookie'

error TS2688: Cannot find type definition file for 'cors'.
  The file is in the program because:
    Entry point for implicit type library 'cors'

error TS2688: Cannot find type definition file for 'karma'.
  The file is in the program because:
    Entry point for implicit type library 'karma'

Here is my package.json.

17

As you are not very familiar with typescript, you can take a shortcut of adding

“skipLibCheck”: true

In tsconfig.json

This will tell typescript to not validate the types of your dependencies

18

That got it compiling yesterday but I opened my project and now the project compiles but I am getting the following error. This is when I try to open the file that contains my custom syntax. I had an issue in my index.ts where somewhere in my git rebasing I lost the part where I actually provide the extension.

Screen Shot 2024-06-22 at 8.16.37 AM
Screen Shot 2024-06-22 at 8.16.37 AM1916×918 178 KB

The only thing different is that my laptop was restarted between yesterdy and today. Do you know if there are any Jupyter scripts or options I can use to find the source of this error?

Thank you so much for your help.

19

How do I use the new CodeMirror theme I create? I added one with the following code but I am not seeing any updates to my syntax highlighting. Do I need to associate the language with the theme?

/**
 * Initialization data for the algorithm theme extension.
 */
export const algorithmtheme_plugin: JupyterFrontEndPlugin<void> = {
  id: 'unicodelab-ts:algorithmtheme',
  autoStart: true,
  // provides: snippetToken, // For providing services
  requires: [IEditorThemeRegistry],
  optional: [],
  activate: (
    app: JupyterFrontEnd,
    theme_registry: IEditorThemeRegistry
  ) => {

    theme_registry.addTheme({
      name: 'algorithm',
      displayName: 'Algorithm',
      theme: [
        EditorView.baseTheme({}),
        syntaxHighlighting(defaultHighlightStyle),
        syntaxHighlighting(HighlightStyle.define([
          { tag: t.number,
            color: "blue",
            textDecoration: "line-through" },
        ]))
      ]
    });
  }
};
20

You should see your new theme in that menu. Click on the name to activate it:

image
image513×544 35.4 KB