Cloud LM Provider

🚀 The [#1](https://github.com/suddhu-iith2004/cloud-lm-provider/issues/1) Extension for AWS Bedrock & Azure OpenAI in VS Code

Bring Claude 4.5, GPT-4o, Nova, DeepSeek, Llama, and 50+ enterprise AI models directly into GitHub Copilot Chat — with intelligent token compression that saves you up to 40% on API costs.

Quick Start • Features • Headroom AI • Models • Configuration • FAQ

🎯 Why Cloud LM Provider?

⚡ Quick Start

Installation

1. Install from VS Code Marketplace

   ext install suddhu-iith2004.cloud-lm-provider
   

   Or search "Cloud LM Provider" in the Extensions sidebar.

2. Run the Configuration Wizard

   Ctrl+Shift+P → "Cloud LM: Manage Provider Configuration"
   

3. Choose Your Provider

   • AWS Bedrock: Enter credentials or use AWS CLI profile
   • Azure OpenAI: Enter endpoint URL and API key

4. Start Chatting

   • Open GitHub Copilot Chat (Ctrl+Alt+I)
   • Select your preferred model from the dropdown
   • Experience enterprise AI in your IDE!

✨ Features

🌐 Multi-Cloud AI Access

Access 50+ enterprise AI models from a single extension:

🎉 What's New in v1.3.5

Azure OpenAI GPT-5 support and smoother deployment management. This release brings GPT-5 deployments into Copilot Chat, fixes the request parameter mismatch for newer models, and makes it easier to add or remove Azure deployments without losing your existing list.

• 🤖 Azure OpenAI GPT-5 support — GPT-5 deployments are now discoverable and usable in Copilot Chat, with the correct max_completion_tokens parameter sent automatically.
• 📝 Preserve deployment list on refresh — When Azure auto-discovery falls back to manual entry, the input is pre-filled with your current deployments so you can edit the list instead of retyping it.
• 🔄 Auto-repair cached deployment metadata — Startup cache entries are re-hydrated with the latest family and capability rules, so stale caches don't hide models after an update.

🎉 Previous Highlights (v1.3.0)

Expanded Azure support and even smarter Headroom compression. This release adds more ways to authenticate with Azure OpenAI and brings additional Azure model deployments into the model picker, while the compression pipeline keeps getting safer and more efficient.

• 🔐 Expanded Azure Authentication & Models — Added support for additional Azure OpenAI authentication flows and new model deployments, giving you more flexibility when connecting to enterprise Azure endpoints.
• 🧠 Headroom Compression Improvements — Further refinements to the compression pipeline for better token savings while preserving the context that matters most.

🎉 Previous Highlights (v1.2.3)

Reliable, mode-aware Headroom compression. We fixed cases where conservative/balanced modes could still drop important tokens, so your models now receive exactly the context they need.

• 🛡️ Mode-aware compression tiers — conservative, balanced, and aggressive now use separate compressor configurations. Conservative mode preserves nearly all non-code context; balanced keeps full function bodies while still summarising very large files and old history; aggressive uses maximum compression.
• 🚫 Current-turn content is fully protected — The active user message and any tool results from the current turn are marked exempt and bypass the pipeline entirely, so the model never receives a stub for content it must understand to answer.
• 🔄 Safer incremental context — Repeated file reads are only deduplicated in older conversation turns; the current turn always keeps the full file content.
• 🛠️ Tool schema compression fixes — Compressed tool definitions are now correctly applied to the outgoing payload, and conservative mode skips tool-description truncation unless context pressure is already high.
• 📊 New per-file exemptions — Right-click any source file in the Explorer and choose Exempt File from Headroom Compression to pass it through unchanged.

🎉 Previous Highlights (v1.2.2)

• 🧠 Adaptive Tool Compression — Dynamic 4-tier compression automatically adjusts based on token budget pressure
• 🖼️ Intelligent Image Compression — Multimodal models now benefit from automatic image format optimization and quality scaling
• 📋 Enhanced Logging — Detailed breakdown of compression metrics, per-content-type savings, and transform tracking with structured [Headroom] format
• ⚙️ Smart Context Caching — Frequently-used context patterns cached and referenced to avoid re-transmission
• 💾 Memory Store Optimization — Improved context deduplication across conversation history
• 🔄 Better Pricing — Fixed cost calculation accuracy with proper compression-induced savings tracking

🧠 Headroom AI — Intelligent Token Compression

Save 25-40% on every API call with our multi-strategy compression engine that never corrupts source code the AI needs to understand:

┌─────────────────────────────────────────────────────────────┐
│                    BEFORE HEADROOM                         │
│  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  21,000 tokens    │
│  (Messages + System Prompts + Tool Schemas)                │
├─────────────────────────────────────────────────────────────┤
│                    AFTER HEADROOM                          │
│  ━━━━━━━━━━━━━━━━━━━━━━━━━  14,700 tokens (-30%)          │
│  (Safe-optimised code + deduplicated + compressed)         │
└─────────────────────────────────────────────────────────────┘

How It Works

1. Safe Source Code Optimisation — Strips comments and normalises whitespace but never modifies logic, variable names, or strings. Preserves TODO/FIXME/HACK and JSDoc @param/@returns. Expected savings: 15-25%.

2. Large File Summarisation — Files exceeding your configured threshold (default 500 lines) are replaced by a structural summary (exports, classes, functions, interfaces, types) plus the first 100 and last 50 lines verbatim. Request specific sections if you need them. Expected savings: 60-80%.

3. Incremental Context Deduplication — When Copilot re-reads the same file in the same conversation, Headroom detects the duplicate and sends a compact reference stub instead of the full content. Expected savings: 40-60% on repeated reads.

4. Adaptive Tool Schema Compression — VS Code injects tool definitions on every request. Headroom intelligently compresses tool descriptions based on available context budget:

   • Conservative mode (< 50% budget): Full descriptions preserved (200-100 char limits)
   • Balanced mode (50-75% budget): Moderate truncation for performance (150-80 char limits)
   • Aggressive mode (75-90% budget): Heavy truncation for cost savings (80-40 char limits)
   • Critical mode (> 90% budget): Maximum compression to stay within limits (50-25 char limits)

   Expected savings: 40-60% depending on aggressiveness level and context pressure.

5. Intelligent Image Compression — For multimodal models, Headroom optimizes image transmission:

   • Automatic format optimization (JPEG, PNG, WebP support)
   • Adaptive image quality scaling based on context pressure
   • Base64 encoding optimization for inline image transmission
   • Fallback image handling for models without vision capabilities
   • Expected savings: 20-35% on image token usage

6. Conversation History Management — Older messages are intelligently summarised while preserving key context and decision points.

7. Smart Context Caching — Frequently-used context patterns are cached and referenced via compact identifiers to avoid re-transmission in long conversations.

Compression Modes

Configuration

{
  "cloudLmProvider.headroom.enabled": true,
  "cloudLmProvider.headroom.aggressiveness": "conservative",
  "cloudLmProvider.headroom.showSavingsNotifications": true,
  "cloudLmProvider.headroom.largeFileThreshold": 500,
  "cloudLmProvider.headroom.exemptCurrentTurnToolResults": true,
  "cloudLmProvider.headroom.exemptTypeDefinitions": true,
  "cloudLmProvider.headroom.exemptFiles": ["src/config/secrets.ts"]
}

Safety Guarantees

• Current-turn content is exempt. The active user message and any tool results from the current turn bypass compression, so the model always sees what it needs to answer.
• Source code is never lossy-compressed. Headroom uses pattern-based stripping only — no LLM rewriting of your code.
• Function bodies are kept in conservative and balanced modes. Code is reduced only by stripping comments and whitespace unless you explicitly choose aggressive mode or the file is very large.
• Compression failures fall back to uncompressed. A broken compression result never reaches the API.
• Tool call/result pairs are validated before and after compression. Orphaned blocks are auto-removed to prevent ValidationException.
• Detailed logging for debugging. Enable cloudLmProvider.headroom.compressionDebugMode to see every exemption, transform, token saving, and context-pressure decision.
• Context-aware adaptive compression. Automatically adjusts compression aggressiveness based on available token budget to prevent exceeding model limits.
• Image integrity preserved. Vision model images are optimized, never corrupted or misrepresented.
• Per-file exemptions. Right-click a file in the Explorer to exempt it from all compression.

Real Savings Dashboard

Track your savings in real-time:

• 💰 Cost Avoided — Exact dollar amounts saved
• 📊 Token Reduction — Session, daily, and lifetime metrics
• 📈 Compression History — Visual trends over time
• 🎯 Accuracy Index — Model attention improvement score

"Headroom saved us $2,400/month across our 50-person engineering team." — Senior Platform Engineer, Fortune 500 Company

Milestone Notifications

Headroom shows a brief toast when you hit a savings milestone:

🎉 Headroom: 1,000,000 tokens saved ($3.00 avoided)    [Open Dashboard]

Milestones fire at every 1M tokens or $5 saved (overall), then auto-dismiss after 5 seconds. Disable via cloudLmProvider.headroom.showSavingsNotifications.

📊 Real-Time Status Bar Telemetry

Always know exactly what you're spending:

$(graph) Tokens: 8,542 In / 1,247 Out | ⚡↓32% | $0.45

• Live token counts from actual AWS/Azure API responses
• Per-request cost calculation using real-time pricing
• Cumulative session tracking for budget management
• One-click dashboard access for detailed analytics

🎯 Smart Notifications & UI

Receive intelligent notifications about your usage and costs:

• 💰 Milestone Notifications — Toast alerts when reaching 1M token or $5 saved milestones
• ⚠️ Cost Warnings — Automatic alerts for expensive model usage
• 📈 Usage Trends — Visual indicators showing compression effectiveness
• 🔔 Status Updates — Real-time feedback on model availability and connection status
• 🎨 Visual Feedback — Compression indicator in status bar showing savings percentage
• 📊 Dashboard Analytics — Comprehensive savings dashboard with daily/lifetime metrics

🔧 Advanced Capabilities

🎛️ Configuration

AWS Bedrock Setup

Option 1: AWS CLI Profile (Recommended)

{
  "cloudLmProvider.aws.defaultRegion": "us-east-1",
  "cloudLmProvider.aws.modelRouting": "auto"
}

The extension automatically uses your configured AWS CLI profile.

Option 2: Access Keys

Run the configuration wizard and enter:

• AWS Access Key ID
• AWS Secret Access Key
• (Optional) Session Token for temporary credentials

Option 3: IAM Role / Instance Profile

For EC2 or ECS environments, credentials are automatically discovered.

Azure OpenAI Setup

{
  "cloudLmProvider.azure.defaultDeployment": "gpt-4o",
  "cloudLmProvider.azure.apiVersion": "2025-01-01-preview"
}

Run the wizard and enter:

• Azure OpenAI Endpoint URL
• API Key or use Azure AD authentication

All Settings

🤖 Supported Models

AWS Bedrock

Azure OpenAI

🔒 Security & Compliance

Cloud LM Provider is built for enterprise environments:

• ✅ No Data Collection — We don't collect, store, or transmit your conversations
• ✅ Local Credential Storage — All secrets stored in VS Code's encrypted keychain
• ✅ Your Cloud, Your Data — Direct API calls to your AWS/Azure accounts
• ✅ SOC 2 / HIPAA Compatible — Works within your existing compliance framework
• ✅ Open Source — Audit the code yourself on GitHub

📈 Performance Benchmarks

Tested on a MacBook Pro M3 with VS Code 1.104:

❓ FAQ

🛠️ Commands

🗺️ Roadmap

• [ ] Prompt Library — Save and reuse effective prompts
• [ ] Team Sharing — Share configurations across your organization
• [ ] Cost Alerts — Configurable spending notifications
• [ ] Google Vertex AI — Support for Gemini models
• [ ] Local Models — Ollama and LM Studio integration
• [ ] Custom Endpoints — OpenAI-compatible API support

🤝 Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

# Clone the repository
git clone https://github.com/suddhu-iith2004/cloud-lm-provider.git

# Install dependencies
npm install

# Compile and watch
npm run watch

# Launch Extension Development Host
F5 in VS Code

📄 License

This project is licensed under the MIT License — see the LICENSE file for details.

🙏 Acknowledgments

• AWS SDK for JavaScript — AWS Bedrock integration
• Azure SDK for JavaScript — Azure OpenAI integration
• Headroom AI — Token compression engine
• The VS Code team for the excellent Language Model API

⭐ If Cloud LM Provider saves you time and money, please star this repo! ⭐

Made with ❤️ by @suddhu-iith2004

📊 Keywords

aws bedrock azure openai github copilot claude gpt-4 llm language model ai assistant code generation token compression enterprise ai vscode extension copilot chat anthropic openai amazon nova deepseek llama mistral cost optimization api cost token tracking headroom context compression