Cloud LM Provider

🚀 The [#1](https://github.com/suddhu-iith2004/cloud-lm-provider/issues/1) Extension for AWS Bedrock & Azure OpenAI in VS Code

Bring Claude 4.5, GPT-4o, Nova, DeepSeek, Llama, and 50+ enterprise AI models directly into GitHub Copilot Chat — with intelligent token compression that saves you up to 40% on API costs.

Quick Start • Features • Headroom AI • Models • Configuration • FAQ

🎯 Why Cloud LM Provider?

⚡ Quick Start

Installation

1. Install from VS Code Marketplace

   ext install suddhu-iith2004.cloud-lm-provider
   

   Or search "Cloud LM Provider" in the Extensions sidebar.

2. Run the Configuration Wizard

   Ctrl+Shift+P → "Cloud LM: Manage Provider Configuration"
   

3. Choose Your Provider

   • AWS Bedrock: Enter credentials or use AWS CLI profile
   • Azure OpenAI: Enter endpoint URL and API key

4. Start Chatting

   • Open GitHub Copilot Chat (Ctrl+Alt+I)
   • Select your preferred model from the dropdown
   • Experience enterprise AI in your IDE!

✨ Features

🌐 Multi-Cloud AI Access

Access 50+ enterprise AI models from a single extension:

🎉 What's New in v1.2.2

Major Headroom AI Enhancements with adaptive context-aware compression and improved observability:

• 🧠 Adaptive Tool Compression — Dynamic 4-tier compression (Conservative, Balanced, Aggressive, Critical) automatically adjusts based on token budget pressure
• 🖼️ Intelligent Image Compression — Multimodal models now benefit from automatic image format optimization and quality scaling
• 📋 Enhanced Logging — Detailed breakdown of compression metrics, per-content-type savings, and transform tracking with structured [Headroom] format
• ⚙️ Smart Context Caching — Frequently-used context patterns cached and referenced to avoid re-transmission in long conversations
• 💾 Memory Store Optimization — Improved context deduplication across conversation history for better efficiency
• 🔄 Better Pricing — Fixed cost calculation accuracy with proper compression-induced savings tracking

🧠 Headroom AI — Intelligent Token Compression

Save 25-40% on every API call with our multi-strategy compression engine that never corrupts source code the AI needs to understand:

┌─────────────────────────────────────────────────────────────┐
│                    BEFORE HEADROOM                         │
│  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  21,000 tokens    │
│  (Messages + System Prompts + Tool Schemas)                │
├─────────────────────────────────────────────────────────────┤
│                    AFTER HEADROOM                          │
│  ━━━━━━━━━━━━━━━━━━━━━━━━━  14,700 tokens (-30%)          │
│  (Safe-optimised code + deduplicated + compressed)         │
└─────────────────────────────────────────────────────────────┘

How It Works

1. Safe Source Code Optimisation — Strips comments and normalises whitespace but never modifies logic, variable names, or strings. Preserves TODO/FIXME/HACK and JSDoc @param/@returns. Expected savings: 15-25%.

2. Large File Summarisation — Files exceeding your configured threshold (default 500 lines) are replaced by a structural summary (exports, classes, functions, interfaces, types) plus the first 100 and last 50 lines verbatim. Request specific sections if you need them. Expected savings: 60-80%.

3. Incremental Context Deduplication — When Copilot re-reads the same file in the same conversation, Headroom detects the duplicate and sends a compact reference stub instead of the full content. Expected savings: 40-60% on repeated reads.

4. Adaptive Tool Schema Compression — VS Code injects tool definitions on every request. Headroom intelligently compresses tool descriptions based on available context budget:

   • Conservative mode (< 50% budget): Full descriptions preserved (200-100 char limits)
   • Balanced mode (50-75% budget): Moderate truncation for performance (150-80 char limits)
   • Aggressive mode (75-90% budget): Heavy truncation for cost savings (80-40 char limits)
   • Critical mode (> 90% budget): Maximum compression to stay within limits (50-25 char limits)

   Expected savings: 40-60% depending on aggressiveness level and context pressure.

5. Intelligent Image Compression — For multimodal models, Headroom optimizes image transmission:

   • Automatic format optimization (JPEG, PNG, WebP support)
   • Adaptive image quality scaling based on context pressure
   • Base64 encoding optimization for inline image transmission
   • Fallback image handling for models without vision capabilities
   • Expected savings: 20-35% on image token usage

6. Conversation History Management — Older messages are intelligently summarised while preserving key context and decision points.

7. Smart Context Caching — Frequently-used context patterns are cached and referenced via compact identifiers to avoid re-transmission in long conversations.

Configuration

{
  "cloudLmProvider.headroom.enabled": true,
  "cloudLmProvider.headroom.aggressiveness": "conservative",
  "cloudLmProvider.headroom.showSavingsNotifications": true,
  "cloudLmProvider.headroom.largeFileThreshold": 500
}

Safety Guarantees

• Source code is never lossy-compressed. Headroom uses pattern-based stripping only — no LLM rewriting of your code.
• Compression failures fall back to uncompressed. A broken compression result never reaches the API.
• Tool call/result pairs are validated before and after compression. Orphaned blocks are auto-removed to prevent ValidationException.
• Detailed logging for debugging. Enable debug mode to see compression breakdown, token savings per content type, context pressure metrics, and transform tracking.
• Context-aware adaptive compression. Automatically adjusts compression aggressiveness based on available token budget to prevent exceeding model limits.
• Image integrity preserved. Vision model images are optimized, never corrupted or misrepresented.

Real Savings Dashboard

Track your savings in real-time:

• 💰 Cost Avoided — Exact dollar amounts saved
• 📊 Token Reduction — Session, daily, and lifetime metrics
• 📈 Compression History — Visual trends over time
• 🎯 Accuracy Index — Model attention improvement score

"Headroom saved us $2,400/month across our 50-person engineering team." — Senior Platform Engineer, Fortune 500 Company

Milestone Notifications

Headroom shows a brief toast when you hit a savings milestone:

🎉 Headroom: 1,000,000 tokens saved ($3.00 avoided)    [Open Dashboard]

Milestones fire at every 1M tokens or $5 saved (overall), then auto-dismiss after 5 seconds. Disable via cloudLmProvider.headroom.showSavingsNotifications.

📊 Real-Time Status Bar Telemetry

Always know exactly what you're spending:

$(graph) Tokens: 8,542 In / 1,247 Out | ⚡↓32% | $0.45

• Live token counts from actual AWS/Azure API responses
• Per-request cost calculation using real-time pricing
• Cumulative session tracking for budget management
• One-click dashboard access for detailed analytics

🎯 Smart Notifications & UI

Receive intelligent notifications about your usage and costs:

• 💰 Milestone Notifications — Toast alerts when reaching 1M token or $5 saved milestones
• ⚠️ Cost Warnings — Automatic alerts for expensive model usage
• 📈 Usage Trends — Visual indicators showing compression effectiveness
• 🔔 Status Updates — Real-time feedback on model availability and connection status
• 🎨 Visual Feedback — Compression indicator in status bar showing savings percentage
• 📊 Dashboard Analytics — Comprehensive savings dashboard with daily/lifetime metrics

🔧 Advanced Capabilities

🎛️ Configuration

AWS Bedrock Setup

Option 1: AWS CLI Profile (Recommended)

{
  "cloudLmProvider.aws.defaultRegion": "us-east-1",
  "cloudLmProvider.aws.modelRouting": "auto"
}

The extension automatically uses your configured AWS CLI profile.

Option 2: Access Keys

Run the configuration wizard and enter:

• AWS Access Key ID
• AWS Secret Access Key
• (Optional) Session Token for temporary credentials

Option 3: IAM Role / Instance Profile

For EC2 or ECS environments, credentials are automatically discovered.

Azure OpenAI Setup

{
  "cloudLmProvider.azure.defaultDeployment": "gpt-4o",
  "cloudLmProvider.azure.apiVersion": "2025-01-01-preview"
}

Run the wizard and enter:

• Azure OpenAI Endpoint URL
• API Key or use Azure AD authentication

All Settings

🤖 Supported Models

AWS Bedrock

Azure OpenAI

🔒 Security & Compliance

Cloud LM Provider is built for enterprise environments:

• ✅ No Data Collection — We don't collect, store, or transmit your conversations
• ✅ Local Credential Storage — All secrets stored in VS Code's encrypted keychain
• ✅ Your Cloud, Your Data — Direct API calls to your AWS/Azure accounts
• ✅ SOC 2 / HIPAA Compatible — Works within your existing compliance framework
• ✅ Open Source — Audit the code yourself on GitHub

📈 Performance Benchmarks

Tested on a MacBook Pro M3 with VS Code 1.104:

❓ FAQ

🛠️ Commands

🗺️ Roadmap

• [ ] Prompt Library — Save and reuse effective prompts
• [ ] Team Sharing — Share configurations across your organization
• [ ] Cost Alerts — Configurable spending notifications
• [ ] Google Vertex AI — Support for Gemini models
• [ ] Local Models — Ollama and LM Studio integration
• [ ] Custom Endpoints — OpenAI-compatible API support

🤝 Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

# Clone the repository
git clone https://github.com/suddhu-iith2004/cloud-lm-provider.git

# Install dependencies
npm install

# Compile and watch
npm run watch

# Launch Extension Development Host
F5 in VS Code

📄 License

This project is licensed under the MIT License — see the LICENSE file for details.

🙏 Acknowledgments

• AWS SDK for JavaScript — AWS Bedrock integration
• Azure SDK for JavaScript — Azure OpenAI integration
• Headroom AI — Token compression engine
• The VS Code team for the excellent Language Model API

⭐ If Cloud LM Provider saves you time and money, please star this repo! ⭐

Made with ❤️ by @suddhu-iith2004

📊 Keywords

aws bedrock azure openai github copilot claude gpt-4 llm language model ai assistant code generation token compression enterprise ai vscode extension copilot chat anthropic openai amazon nova deepseek llama mistral cost optimization api cost token tracking headroom context compression