flexiable-gpt-local README

Flexiable-gpt-local is designed for authors to help with writing blogs, briefs, articles, and books in markdown or plaintext files. Similar to pay apps like Rytr, but without the monthly membership. To do that this extension provides some basic, core prompts that can be bound to keyboard shortcuts like "expand" and "concise". But given that everyone has different styles and needs, this plugin allows you to build your own canned prompts via configuration. See below for details

All of these prompts work based on selected text. They can operate in replace/rewrite, prefix, or postfix mode. Prefix inserts generated content before the selected text, and postfix inserts it after.

This plugin is compatible with any OpenAI compatible backend, and is configured to use OpenAI's API's by default. If you do not intend to run LLM's locally, this means you will need an Open AI token and a positive ballance in your openAI account.

Features

AI backend support

AI Backend Supported	Details
OpenAI compatible API's	Requires a pay as you go account and a token to be provided from their api key page

Built in AI Generation commands

All of these commands operate on selected text unless otherwise noted.

Command Name	Description
Expand	Take the selected text and expand upon it. This will replace the selected text. It will not work without a selection.
Condense	Take the selected text and condense it by making it more concise. This will replace the selected text. It will not work without a selection.
Alter Tone	Rewrite the selected text with a particular tone. This will open up a quick pick window of available tones.

I am trying to balance the overhead of having more commands with the power of shortcut keys for commonly used commands and reducing noise in the command pallet by multiplexing commands with quick edits, like tone editing. That said, I welcome suggestions and features to add to the built-in command set.

Frontmatter

This plugin leverages frontmatter for metadata and to power several features. Ths includes managing context and storing metadata.

Elements and context

This plugin can load up context based on references and full_references in the frontmatter of the document. This allows you to control precisely the scope of context that you pass to the LLM by easily pulling in reference information, previous chapters, character histories, and more.

Place reusable content that you wish to reference inside of a folder called elements that is located in any parent directory relative to where you are working. All elements folders will be scanned for matching references and they will all be included. This structure allows you to have elements that are global, like your writing style and world history. But also to have elements specific to an individual book. Wildcards are also supported.

Below you will see an example of a wildcard input for references and a specific import for full_references. When your context becomes large, references will use their summary instead of their raw content. full_references will always use the full content of the file when building a context. Please be mindful for full_references as they can significantly increase your context size.

---
references: 
  - /PlotDevice/*
full_references:
  - /Meta/BeatSheet.md
---

Simply supply the path to a particular md file from the root of an elements folder.

Hint: Use folder naming conventions that are unique for your shared or root elements folder to avoid conflicts.

If you have a lot of context, the summary field of the frontmatter may be used rather than the content of the element itself to reduce context size.

Summaries

Frontmatter can store a special summary field that is used to store or cache a summary of the document you are editing. You can update it with the command flexiable-gpt-local.summarize. If a summary isn't present, it will be generated when that file is used as a reference.

Managing costs

For those using a pay-per-token API like OpenAI/ChatGPT, deepseek, or another openAI compatible API, you may want to manage your costs. Here are some major ways to manage your costs.

Setup limits with your API provider to limit costs. Avoid "auto-refill" for you balances to ensure that you don't accidentally cost yourself big.
Mange the size of your context. Check your frontmatter for what will be included in your context and prefer to use references over full_references to manage this directly.
With each interaction with the API, you will see a notification for both the input token count (Context and input prompt) and the output tokens. These numbers are how API's bill your account.

Commands For Changing Settings

Command Name	Description
Set Context	Set the "context" that is given to the AI model for the current workspace. If you have text selected when you run this action, just the selected text is loaded into the context. Otherwise, the whole active document is loaded as context.

Context is simply the first system message in a chat exchange. Consider it reusable instructions about how you want the AI to respond, context about what you want to do, or helpful data for it to answer questions about. Read this blog for more details about context.

Common use cases for context:

Describing the expectations on generated text. I.e. setting the tone and voice of the author.
Giving context around what you are writing. I.e. an outline of a larger book or summaries of previous chapters.
Raw information. I.e. A scholarly (aka dense) article or academic paper you want to summarize or focus your piece on.

Please keep in mind that all models limit total input tokens per request. This includes the context, the prompt you use, and the text you have selected. They also charge for it! Consider condensing your context to reduce it's size.

Custom prompts

Command Name	Description
Manual	This allows you to specify a behavior (replace selected text, prefix selected text, or postfix selected text) and then type in an instruction. For when you intend to use a prompt only once or infrequently.
Replace Manual	Since most commands issues are replace's, this is a shortcut for a manual replacement. Its valid to not enter a prompt, in which case the selected text will be used as the prompt/instruction.
Custom	Select from the configured prompts by name.

If you find yourself using similar manual prompts over and over again, put them into your settings. It does require digging into the JSON configuration for this plugin, but there are examples there to help. The options for behavior are "replace", "prefix" and "postfix".

If you are getting really good results out of a custom prompt, consider sharing it on GitHub as a feature request.

Auditing and debugging

If auditing is enabled using the flexiable-gpt-local.auditing config, each API request will be logged to the /.audit in the root of your open directory. Each audit file is a YAML representation of the request and response from your LLM in its raw format. This is great for debugging, but it can eat up diskspace over time. You can simply delete the directory as needed to free up disk space, or disable the feature if you don't need it.

Extension Settings

Please consult the configuration section of the package.json.

Known Issues

This plugin hasn't included the streaming API yet.
Making edits to the file while an AI powered edit is pending is a bit finicky. Its best to wait for the completion to finish before making other edits.

Want to help?

I freely admit that I am neither a Typescript Wizard nor an experienced extension developer. If you find this helpful and want to contribute your coding skills, check out the open issues on github. If you have ideas for how to improve this extension, but don't have the coding skills to do it, open up a github issue and tag it with "enhancement".

You can also consider becoming a sponsor or leaving a tip.

Release Notes

0.1.0

Following extension guidelines

Ensure that you've read through the extensions guidelines and follow the best practices for creating your extension.

Extension Guidelines

For more information

Enjoy!

flexiable-gpt-local

Erik Helleren