NEOSIS — Code Understanding Validator

A VS Code extension that watches you code and quizzes you on your own changes using adaptive multiple-choice questions generated by OpenRouter AI.

How it works

File save / keystroke
  └─► SmartTrigger (15 s debounce, 30 s minimum gap between sessions)
        └─► CodeAnalyzer  — extracts functions, classes, variables, language, complexity
        └─► ChangeDetector — diffs current vs. previous snapshot; classifies changes
              (function_added, condition_change, loop_change, error_handling_change, …)
              Major changes (new function / class, ≥25 % size shift) bypass the gap timer
        └─► QuestionEngine (OpenRouter) — builds an adaptive prompt and calls the API
              Prompt encodes: difficulty level, weak topics, detected changes, code snippet
              Falls back to deterministic offline questions on network/API failure
        └─► Webview panel — renders the quiz; user answers feed back into PerformanceTracker
              └─► Evaluator — grades MCQ answers (full/partial/wrong)
              └─► PerformanceTracker — updates score, streak, weak-area map, retry queue

Core modules

Prerequisites

• VS Code ≥ 1.73
• Node.js ≥ 18 (needed only for npm install)
• An OpenRouter API key — free tier available at https://openrouter.ai/keys

Setup

1. Install dependencies

npm install

2. Set your API key

Option A — .env file (picked up automatically at extension start):

# .env (already in .gitignore)
OPENROUTER_API_KEY=sk-or-v1-your-key-here

Option B — VS Code Settings (survives across workspaces):

Open Settings (Ctrl+,), search for NOESIS, and paste your key into Code Validator: Openrouter Api Key.

The extension checks VS Code settings first and falls back to OPENROUTER_API_KEY in the environment / .env file.

3. Launch in debug mode

Press F5 in VS Code. A new Extension Development Host window opens.

Usage

Quiz mechanics

• Questions are multiple-choice (4 options) generated for the specific code on screen.
• Difficulty adapts automatically: accuracy > 80 % → hard, 40–80 % → medium, < 40 % → easy.
• Wrong answers enter a retry queue; up to 2 are re-served at the end of each session.
• A streak bonus (+10 pts) fires every 3 consecutive correct answers.
• Rapid correct answers (< 10 s) earn a time bonus; slow wrong answers (> 30 s) incur a penalty.
• Topics where accuracy drops below 60 % are flagged as weak areas and weighted higher in future prompts.

Commands

Supported languages

JavaScript, TypeScript, Python, Java, C++, C, Go, Rust (configurable via codeValidator.supportedLanguages).

Configuration

All settings live under the codeValidator namespace in VS Code Settings:

Question types and scoring

Base points per grade: correct → 10, partial → 5, wrong → 0 (plus a −5 penalty). Confidence modifier: high confidence + wrong answer subtracts an additional 10 pts.

Offline fallback

If the OpenRouter API call fails (network error, rate limit, missing key), QuestionEngine generates deterministic questions from the local code analysis — covering the same detected changes (new functions, condition changes, loops, etc.) with no external dependency. These questions carry the same schema as API-generated ones.

Project structure

noesis/
├── src/
│   ├── extension.js              Entry point — activates all modules
│   ├── codeAnalyzer.js           Static code analysis (language-agnostic regex)
│   ├── storage.js                File-backed persistence for questions / stats
│   ├── webviewContent.js         Quiz UI (HTML + inline CSS/JS)
│   ├── ai/
│   │   └── questionEngine.js     OpenRouter API client + offline fallback
│   ├── analyzer/
│   │   └── changeDetector.js     Structural diff engine
│   ├── evaluator/
│   │   ├── evaluator.js          Grading logic (MCQ + free-text)
│   │   └── performanceTracker.js Score / streak / weak-area / retry-queue state
│   └── trigger/
│       └── smartTrigger.js       Debounce + minimum-gap gating
├── .env                          API key (gitignored)
├── .env.example                  Template
├── package.json
└── README.md

Troubleshooting

No questions generated

• Check the Debug Console (F5 → Developer Tools) for [QuestionEngine] log lines.
• If you see OPENROUTER_API_KEY not found, add the key to .env or VS Code Settings.
• The extension falls back to offline questions automatically; those still display normally.

Quiz panel doesn't open

• Make sure the file is at least 30 characters and in a supported language.
• Try the status-bar NOESIS button to force a fresh run.

"Cannot find module"

• Run npm install in the repo root.

License

MIT