How to add new autocompletions

jaysun_n · June 11, 2021, 2:26pm

I want to add autocompletions for unicode characters (like ‘->’ becoming ‘→’) so I can use it similar to a shortcut, but cannot find out how to implement this functionality as a JupyterLab extension. I have read the docs on the completer module, but it doesn’t seem to be able to do what I want. I have also tried googling for an answer and looking through Jupyter LSP github, but have been unsuccessful. Are there any examples, templates or explanations on how to add new autocompletions?

krassowski · June 11, 2021, 2:36pm

This depends on the language/kernel that you are using. For example, if you use IPython you could do something like this:

from IPython.core.latex_symbols import latex_symbols

# >l<eft a>r<row (okay, I could have chosen a better alias)
latex_symbols['\\lr'] = '→'

See https://stackoverflow.com/a/50647764/6646912

jaysun_n · June 11, 2021, 2:52pm

I am making the extension in typescript and want to make it not specific to any language or notebook that is opened.

Also, as mentioned in my other question, the end goal is to add these types of auto completions to markdown cells and it doesn’t seem like IPython shortcuts can work in the markdown cells.

krassowski · June 11, 2021, 4:32pm

Oh, that’s a completely different situation. Sorry I did not realise you are writing an extension. It is not currently easy I am afraid. I will be working to make it possible (see Refactor completer to support external providers by krassowski · Pull Request #600 · krassowski/jupyterlab-lsp · GitHub and Completer public API refactor wishlist · Issue #9763 · jupyterlab/jupyterlab · GitHub) in July/August but this will only arrive for JupyterLab 4.0. If you absolutely have to have this functionality, then in the meantime you would likely need to follow the same path as jupyterlab-lsp.

jpcaram · November 17, 2023, 3:10pm

Hi @krassowski, I’ve been trying to learn how to create custom completers (by IPython, and do want it per notebook as opposed to @jaysun_n) and it’s been quite a struggle. IPython’s documentation (IPython Completer) on this is not good (barely anything more than API docs, and those are bad too). There aren’t any good examples elsewhere either. It also seems there are different APIs for different versions of IPython with strange compatibility issues between them.

Can you point me to any good resources, examples, or whatever, on the subject?

My specific goal is similar to the Latex example you provided. I want to have some keyword that is replaced by a suggested completion. The replacement is important, so it’s not really a “completion”.

I’ve managed to implement something that works:

from IPython.core.completer import context_matcher, CompletionContext, SimpleCompletion
import re

@context_matcher()
def custom_matcher(context: CompletionContext):
    completions = []
    m = re.search(r"^_p(.*)", context.token)
    if m:
        completions.append(
            SimpleCompletion(f"jobs = get_jobs("{m.group(1)}"), type="MyStuff")
        )
    return dict(completions = completions, suppress = True)

get_ipython().Completer.custom_matchers.append(custom_matcher)

But I get different behavior depending on the versions of IPython and Jupyter, and whether I’m directly in IPython or in a notebook.

I would also like to fully customize a completer (or it is a “matcher”? I don’t know what I’m doing). I guess I would need to define a class (Inherit from what?) instead of the @context_matcher() decorator, and perhaps return something other than SimpleCompletion? I would also like to customize the tokenization. Etc, etc.

Is there something I need to implement differently for completions in notebooks versus IPython console?

Nowadays we have lsp. Should we be implementing these things through different lsp servers instead of IPython?

IPython’s documentation mentions Jedi. Should I be looking at Jedi’s ways instead?

Sorry for the long post. I’d be happy to start another thread if that will help other better. Thanks!

krassowski · November 25, 2023, 1:19pm

IPython’s documentation (IPython Completer) on this is not good (barely anything more than API docs, and those are bad too). There aren’t any good examples elsewhere either.

Ouch. Contributions are welcome feel free to tag me for a review. Though any examples would need to be maintained by someone and I did not see too many folks volunteer to contribute here

It also seems there are different APIs for different versions of IPython with strange compatibility issues between them.

What do you mean specifically? Are there any incompatibilities that you found? Or do you just mean that we added an arguably a better API which was not available in older IPython versions?

I’ve managed to implement something that works:

Looks good I guess? There is a missing " in your code I believe. Using regex on a token is a bit naive for anything more complex, the intent of exposing token is not that it will solve all your problems but is good enough for 80% of custom completion matchers.

But I get different behavior depending on the versions of IPython and Jupyter, and whether I’m directly in IPython or in a notebook.

Which is?

I want to have some keyword that is replaced by a suggested completion. The replacement is important, so it’s not really a “completion”.

That should work, except that depending on what extensions you have installed (wink jupyterlab-lsp) pre-filtering may be enabled rejecting compeltions which do not start with the prefix; this will be fixed in jupyterlab-lsp in the future.

I would also like to customize the tokenization.

You may want to have a look at:

github.com

ipython/ipython/blob/c293abc5cebd8d3ee1a09221043a88a8f8ac7d39/IPython/core/completer.py#L2520-L2635


      
          def dict_key_matches(self, text: str) -> List[str]:
              """Match string keys in a dictionary, after e.g. ``foo[``.
          
              .. deprecated:: 8.6
                  You can use :meth:`dict_key_matcher` instead.
              """
          
              # Short-circuit on closed dictionary (regular expression would
              # not match anyway, but would take quite a while).
              if self.text_until_cursor.strip().endswith("]"):
                  return []
          
              match = DICT_MATCHER_REGEX.search(self.text_until_cursor)
          
              if match is None:
                  return []
          
              expr, prior_tuple_keys, key_prefix = match.groups()
          
              obj = self._evaluate_expr(expr)

This file has been truncated. show original

github.com

ipython/ipython/blob/c293abc5cebd8d3ee1a09221043a88a8f8ac7d39/IPython/core/completer.py#L1129-L1194


      
          def attr_matches(self, text):
              """Compute matches when text contains a dot.
          
              Assuming the text is of the form NAME.NAME....[NAME], and is
              evaluatable in self.namespace or self.global_namespace, it will be
              evaluated and its attributes (as revealed by dir()) are used as
              possible completions.  (For class instances, class members are
              also considered.)
          
              WARNING: this can still invoke arbitrary C code, if an object
              with a __getattr__ hook is evaluated.
          
              """
              m2 = re.match(r"(.+)\.(\w*)$", self.line_buffer)
              if not m2:
                  return []
              expr, attr = m2.group(1, 2)
          
              obj = self._evaluate_expr(expr)

This file has been truncated. show original

github.com

ipython/ipython/blob/c293abc5cebd8d3ee1a09221043a88a8f8ac7d39/IPython/core/completer.py#L2319-L2340


      
          @completion_matcher(api_version=1)
          def python_matches(self, text: str) -> Iterable[str]:
              """Match attributes or global python names"""
              if "." in text:
                  try:
                      matches = self.attr_matches(text)
                      if text.endswith('.') and self.omit__names:
                          if self.omit__names == 1:
                              # true if txt is _not_ a __ name, false otherwise:
                              no__name = (lambda txt:
                                          re.match(r'.*\.__.*?__',txt) is None)
                          else:
                              # true if txt is _not_ a _ name, false otherwise:
                              no__name = (lambda txt:
                                          re.match(r'\._.*?',txt[txt.rindex('.'):]) is None)
                          matches = filter(no__name, matches)
                  except NameError:
                      # catches <undefined attributes>.<tab>
                      matches = []
              else:

This file has been truncated. show original

Nowadays we have lsp. Should we be implementing these things through different lsp servers instead of IPython?

No. LSP will not give you completions in IPython

IPython’s documentation mentions Jedi. Should I be looking at Jedi’s ways instead?

No. Jedi has a scope for helping with code completions not emmet completions AFAIK.

Topic		Replies	Views
Autocompletions in markdown cells extensions how-to , help-wanted	4	1561	September 30, 2022
Autocompleter Extensions in extension-examples github extensions jupyterlab	1	477	August 7, 2022
Enhanced autocompletion in Julia, R and Python (LSP) extensions	11	1663	September 18, 2023
Editor customization JupyterLab how-to	0	359	April 7, 2020
Jedi Autocompletion disappears after typing the first word JupyterLab jupyterlab , how-to , help-wanted	3	812	September 2, 2021

How to add new autocompletions

Related Topics