token-estimator

A small VS Code extension that estimates token usage for prompts and code snippets, helpful for LLM cost planning and debugging. It shows token counts and cost estimates for selected text, inputs in the webview, or entire files.

Features

• Estimates token counts for selected text or open documents.
• Estimates cost using configurable model pricing.
• Lightweight webview UI for quick checks.
• Keyboard command to evaluate selection or active editor.
• Development watch tasks for TypeScript and bundling.

Quick Start

• Install dependencies:

    npm install
  

• Run the extension in the VS Code Extension Development Host:

  • Launch the debugger from VS Code (Run Extension).

  • Or run the provided watch tasks:

      npm run watch
    

Usage

• Open a file or select text in the editor.
• Run the command Token Estimator: Estimate tokens from the Command Palette, or open the extension's webview if available.
• The webview displays token count and estimated cost; you can paste any prompt into the input box to test.

Extension Settings

This extension contributes optional settings. Update the keys to match your package.json if different.

• tokenEstimator.model: Model name used for pricing, for example gpt-4o-mini.
• tokenEstimator.pricePerKTokens: Price per 1,000 tokens in USD.
• tokenEstimator.showNotifications: Show editor notifications on estimate.

Example (settings.json):

{
	"tokenEstimator.model": "gpt-4o-mini",
	"tokenEstimator.pricePerKTokens": 0.03,
	"tokenEstimator.showNotifications": true
}

Development

• Build/watch:

    npm run watch
  

• Tests:

    npm test
  

• Key files:

  • Main extension entry: [src/extension.ts]
  • Tokenizer/logic: [src/tokenizer.ts]
  • Pricing logic: [src/pricing.ts]
  • Webview provider: [webview/TokenEstimatorViewProvider.ts]

Release Notes

1.0.0

• Initial release: token counting, cost estimation, webview UI, editor commands.

Contributing

• Open issues for feature requests and bugs.
• Create pull requests for fixes or improvements; run tests and linters locally before submitting.

License

Specify your license here, for example MIT.