llama.cpp Token Cost Tracker

Provide token usage and estimated costs for llama.cpp deployments. Also useful for estimating savings when using local LLMs :) The extension acts as a HTTP proxy to intercept activity from other extensions that interact with llama.cpp such as RooCode, Continue, etc.

Source Code

github.com/tudoraneau/llamacpp-token-cost-tracker

Disclaimer

Use at your own risk, no warranties provided, express or implied. This extension is not affiliated in any way with any other projects.

Price Calculation Logic

inputPricePerToken = inputCostPerMillion / 1,000,000
outputPricePerToken = outputCostPerMillion / 1,000,000
inputCost = promptTokens * inputPricePerToken
outputCost = completionTokens * outputPricePerToken
totalCost = inputCost + outputCost

Requirements

• VS Code 1.85.0 or higher
• llama.cpp server running

Setup

1. Configure the llama.cpp server URL under llama.cpp Configuration (default: http://localhost:8080). The hostname/IP address must match the machine where llama.cpp is running.
2. Ensure the proxy port is available on the local machine where this extension is running (default: 8081)
3. Configure the proxy port to 8081 under Proxy Settings
4. Configure the proxy target URL to the server URL running the llama.cpp server (http://localhost:8080). The hostname/IP address must match the machine where llama.cpp is running
5. Configure cost settings
6. Point your other extension (i.e.: RooCode, Continue) to the URL where the proxy is running (http://localhost:8081/v1)

Settings

• tokenTracker.currency: Currency for cost calculations (default: USD)
• tokenTracker.decimalPlaces: Decimal places for cost calculations (default: 4)
• tokenTracker.maxHistoryEntries: Maximum number of history entries (default: 100000)
• tokenTracker.showStatusBar: Show status bar item (default: true)
• tokenTracker.confirmBeforeDelete: Confirm before deleting data (default: true)
• tokenTracker.storageScope: Storage scope for data (default: global)
• tokenTracker.proxy.enabled: Enable proxy mode for token tracking (default: true)
• tokenTracker.proxy.port: Port for proxy service (default: 8081)
• tokenTracker.proxy.targetUrl: Target URL of llama.cpp server (default: http://localhost:8080)
• tokenTracker.llamaCpp.serverUrl: URL of llama.cpp server (default: http://localhost:8080)
• tokenTracker.llamaCpp.requestTimeoutMs: Request timeout in milliseconds (default: 5000)
• tokenTracker.llamaCpp.enableLogMonitoring: Enable log monitoring for token tracking (default: false)
• tokenTracker.llamaCpp.logPath: Path to log file for monitoring

Commands

• Token Tracker: Open Dashboard
• Token Tracker: Export JSON
• Token Tracker: Export CSV
• Token Tracker: Import JSON
• Token Tracker: Import CSV
• Token Tracker: Reset Session
• Token Tracker: Clear History
• Token Tracker: Backup Database
• Token Tracker: Restore Database
• Token Tracker: Select Log File
• Token Tracker: Stop Log Monitoring

License

MIT