This extension does not come with a built in backend for running LLMs. Instead you are able to use any existing tool that supports the OpenAI API format. Like the Oobabooga WebUI and many other
Features
Inline (multi line) code completion
Works with any OpenAI compatible API
Save multiple API Endpoints and switch easily between them
Reducing requests to LLMs by
saving previous responses
skiping completion depending on the last symbol
only posting request if no input was given for some time (can be specified in the settings)
Dynamically detect multi line or single line completion
Roadmap
Improve (optional) features to reduce LLM requests
Add option (possible regex) to specify after which characters the LLM should be/not be triggered
Increase context
Add content after cursor to prompt
Add content of other files
Return multiple completions (add suggestions from history)
Improve detection of already existing symbols at the end of a completion
Reduce chance of repeating already existing symbols
Reduce completion stopping because of false detection of already existing symbols
Extension Settings
localcompletion.active_endpoint: The URL of the API which is used for generating the code completion
localcompletion.endpoints: List of URL endpoints
localcompletion.temperature: Temperature of the LLM
localcompletion.max_tokens: Maximum number of tokens in the response
localcompletion.reduce_calls: Reduce API calls with various strategies (e.g. skip completion if last symbol was a letter)
localcompletion.skip_autocomplete_widget: Skip completion if autocomplete widget is active
localcompletion.completion_timeout: Minimum time between keystrokes (in ms) before sending a completion request (Reduces API calls, which are closed immedietly after)
Known Issues
The extension does not yet support a custom API key. This means it only works for APIs which do not need a key.
Model switching is not supported at the moment as most local tools don't support that property either.
Release Notes
0.0.5
Increase time between keystrokes before requesting a new completion
Show inline completion even if autocomplet widget is active (can be disabled)
Fix bug where sometimes a running completion would not be stopped if a new completion is triggered
0.0.4
Distinguish between single line and multiline completion by checking text after the cursor
Add '\n' to stop token for single line
Reduce repetition of already existing symbols (like '}' or ';') at the end of a completion
Remove completion from history for new line (most predictions where totally wrong)
0.0.3
Rework handling of old responses
Only call new completion if input deviates from previous completion