Ai Code Checker 🐿️

This extension provides an ai code checker for simple code checking. It is intended for JavaScript but can probably be used for other languages as well.

Features

It adds a button with a squrril 🐿️ icon to the top right corner of the editor window. When you click the button, the extension will check the code in the editor window and display problems it finds as squiggly lines under the code. When you hover over the squiggly lines, a tooltip will display the problem.

This is all done by an AI model that you prompt yourself in a prompt.md file in the .vscode folder. The AI model will output JSON that the extension will use to display the problems.

There can be different problem types defined in the prompt for the AI model. Each problem can have a unique color for the squiggly lines defined in the settings.

Setup

1. Choose an API host that works with the OpenAI API format. (e.g. openrouter)
2. Set the url and api key of the host as well as the model you want to use in the extension settings
3. Open any project and add a prompt.md file to the .vscode folder. This is the prompt for the AI model.
4. Reload the window and click the button in the top right corner of the editor window to check the code.

Note

All comments and empty lines will be removed before the code is sent to the AI model.

The model will get the code with xml-like line numbers inserted before each line. There is a permanent built-in instructin to inform the model about this. With the setting aiCodeChecker.aiModelGetsCodeTwice the model will also get the code without line numbers.

This extension will request the model through an api flag to only output JSON. For this to work with openai models, you will have to instruct the models in the prompt to output JSON too. This is the format in which the extension expects the LLM output:

{
  "type": "object",
  "properties": {
    "line": {
      "type": "integer",
      "description": "Line number of the error"
    },
    "original": {
        "type": "string",
        "description": "Original code that caused the error. Can be a single word or the whole line."
    },
    "type": {
        "type": "string",
        "description": "Type of the error. Can be spelling, syntax, etc. You can define your own types."
    },
    "friendly": {
        "type": "string",
        "description": "Friendly error message that will be displayed to the user."
    }
  },
  "required": ["line", "original", "type", "friendly"]
}

Example JSON:

{
  "line": 1,
  "original": "colour",
  "type": "spelling",
  "friendly": "`colour` ist kein Turtle Befehl. Meinst du vielleicht `color`?",
}

Extension Settings

This extension contributes the following settings:

• aiCodeChecker.apiKey: API Key corresponding to the url of the used endpoint
• aiCodeChecker.modelName: AI model name
• aiCodeChecker.apiUrl: URL for API endpoint in openai format
• aiCodeChecker.colors: Colors per category of error for squiggly lines. This should match the categories in the prompt for the AI model.
• aiCodeChecker.aiModelGetsCodeTwice: If true, the AI model will get the original code (only comments removed) additionally to the code with inserted line numbers.

Example prompt.md

# Task
The input is a simple standalone JavaScript program. Your task is to output JSON that describes scefic problems with the code. The code may use turtle commands.
# Possible Turtle commands
`forward(length)`, `left(angle)`, `right(angle)`, `penup()`, `pendown()`, `color(color)`, `color(r, g, b)`, `goto(x, y)`, `clear()`, `angle(angle)`, `width(width)`, `showGrid()`, `randomColor_h()`, `strangeLine(length)`, `strangeSquare(length, step)`, `strangeCircle(radius)`, `strangeGalaxy(radius)`, `setSpeed(speed)`
# Error types
Report an error for any of the following:
## Spelling
- misspelled turtle command (`forwad(10)`)
- variable that is used with a different spelling than it was defined (`var linienFarbe = "red"; color(linienfarbe);`)
- misspelled JavaScript keyword or internal (`arr.lenght;`)
## Syntax
- missing `()` in a function call, an if statement or a loop (`forward 10;`)
- missing `"`/`'` around what is supposed to be a string (`color(red);`)
- for loop with `,` instead of `;` (`for (var i = 0, i < 10; i++)`)
- wrong syntax in any while-loop, for-loop, if-statement, function definition (`while { i++; (i < 10)}`)
## Semantic
- variable with name of a turtle command (`var forward = 10; forward(10);`)
- wrong number of arguments for a turtle command (`forward(10, 20);`)
- negative number in forward or width command (`forward(-10);`)
- using color with a non-english color name (`color("rot");`)
- non-booleans as conditions in if, loops (`let x = 4; if (x)`)
- assinment of turtle function result to a variable (`var x = forward(10);`)
- calculation without assignment (`let x = 5; x + 1;`)
- a user defined function that is never called
- an infinite loop (`for (var i = 0; i < 10; i--)`, `for (var i = 10; i > -1; i++)`)
- an assignment in a condition (`if (x = 5)`, `for (var i = 0; i = 10; i++)`)
- an if-statement/loop without curly braces (`if (x > 5) forward(10);`)
## Style
- using any combination of `left`, `right` and `angle` without a `forward` in between (`left(90); right(30);`)
- using any of `goto`, `color`, `width`, `penup`, `pendown` twice without a `forward` in between
- using `clear` directly after something is drawn (`forward(10); clear();`)
- an empty if-statement with a non-empty else block (`if (x > 5) {} else { forward(20); }`)
- the exact same code line/lines repeated 5 times or more diretly one after the other without using a loop (`forward(10); forward(10); forward(10); forward(10); forward(10);`; only mark first appearance as error)
# Ignore errors
Do not report any other errors than the ones listed above! For example, ignore any missing or inconsisent use of semicolons, indentation, etc.
# Output format
Output nothing but JSON. The JSON object must have a single key `errors` that is a list of all errors. Each error is an object like this:
```json
{
  "line": 1,
  "original": "colour",
  "type": "spelling",
  "friendly": "`colour` ist kein Turtle Befehl. Meinst du vielleicht `color`?",
}
```
Any error object must represent one of the problems defined above with the line number of the error, the relevant part of the line that contains the error as it appears in the code, the type of the error as defined above and a simple, beginner-friendly german explanation of the problem. Remember to escape quotes in the JSON output.

Enjoy!