Naico AI Copilot Tracker
A VS Code extension that tracks and analyzes GitHub Copilot usage with detailed reports and metrics.
CONFIDENTIALITY NOTICE
NAICO ITS CONFIDENTIAL & PROPRIETARY
This software and its documentation contain proprietary information of NAICO ITS.
It is provided under a license agreement containing restrictions on use and disclosure.
Unauthorized reproduction, distribution, or disclosure is strictly prohibited.
All rights reserved. © NAICO ITS.
Features
- Automatic Tracking: Always-on monitoring of Copilot-generated code
- Detailed Analytics: Daily and weekly usage statistics
- Language Breakdown: See which programming languages benefit most from Copilot
- Billing Savings: Calculate financial savings based on developer hourly rates
- Productivity Metrics: Estimate time saved with AI assistance
- Exportable Reports: Download CSV reports of your Copilot usage
- Team Performance: Compare productivity across your organization
Installation
Install from VS Code Marketplace:
- Search for "NaicoITS".
- Select Naico AI Copilot Tracker
- Install.
Manual Installation via VSIX:
- Download the latest VSIX file from the Releases Folder
- In VS Code, go to Extensions view (Ctrl+Shift+X)
- Click on the "..." at the top of the Extensions view
- Select "Install from VSIX..."
- Navigate to your downloaded VSIX file and select it
Setup
Enable the Extension:
- After installation, the extension will be enabled by default
- You can verify by checking the status bar for the Copilot tracking indicator
Configuration:
- Open VS Code settings (Ctrl+,)
- Search for "Naico AI Copilot"
- Customize tracking preferences according to your needs
- Available settings include:
naico.aiCopilotTracker.enabled
: Enable/disable the extension
naico.aiCopilotTracker.trackingMode
: Set the tracking level
naico.aiCopilotTracker.developerRate
: Set your hourly rate for cost savings calculations
Usage:
- The extension automatically tracks your GitHub Copilot interactions
- View your statistics in the dedicated Copilot Tracker panel
- Set your developer hourly rate to calculate financial savings
- Export reports as needed through the panel options
Quick Start
- Set your GitHub username via Command Palette:
Set GitHub Username
- Use Copilot as normal - tracking starts automatically
- Open the Naico ITS sidebar to view stats
- Access the report in multiple ways:
- Click the graph icon in the status bar
- Use the "View Report" button in the Naico ITS sidebar
- Run the "View Copilot Tracking Report" command from the Command Palette
Dashboard Features
- Tracking Status: Shows active monitoring status
- Manual Refresh: Update stats on demand
- Multiple Views: Switch between Productivity, Daily, Weekly, and Billing Savings views
- Developer Rate Setting: Enter your hourly rate to calculate exact financial savings
- Export Options: Download usage reports by date or month
- Team Comparison: View how your usage compares to other team members
MongoDB Data Upload
For organizations that want to centralize Copilot usage tracking, the extension includes a MongoDB upload feature:
- One Entry Per User Per Day: Optimized data structure for organizational analytics
- Centralized Tracking: Upload usage statistics to a shared MongoDB database
- Team Insights: Compare productivity across developers on your team
- Simple Access: Upload data with one click using the
Upload Data to MongoDB
command
- Pre-Configured Connection: Uses a hardcoded connection string to Naico ITS MongoDB Atlas cluster
- Data Auto Sync feature: Syncing data with MongoDB every 10 mins
How to Use MongoDB Upload
- Run the
Upload Data to MongoDB
command from the Command Palette.
- Data for the current day will be uploaded to the database.
- Check the output channel for success or error messages.
Organization Reports
After data is uploaded to MongoDB, you can access organization-level reports:
- Open the Copilot Tracking Report
- Navigate to the "Organization" tab
- Enter the organization password when prompted
- View consolidated metrics including:
- Total time saved across your team
- Lines of code generated by all developers
- Cost savings based on developer rates
- Active developer count and performance
- Download monthly organization reports in CSV format for further analysis
Troubleshooting
If charts aren't displaying:
- Click the Refresh button
- Switch between tabs
- For persistent issues, add these settings to your VS Code settings.json:
"naico-ai-copilot-tracker.chartColors": {
"accentColor": "#0078D4",
"borderColor": "#444444",
"textColor": "#CCCCCC",
"bgColor": "transparent"
}
Commands
START Naico AI
: Start tracking (if not already running)
View Copilot Tracking Report
: Open analytics dashboard
Download Copilot Usage Report
: Export data as CSV
Download Monthly Report
: Export monthly organization data
Set GitHub Username
: Configure your identity
Upload Data to MongoDB
: Share your stats with your organization
Data Privacy
- All data stored locally on your machine
- No data sent to external servers unless you explicitly use the MongoDB upload feature
- You control and can delete your data at any time
- Organization data is stored securely in MongoDB Atlas
How It Works
The extension tracks:
- Lines of code generated by GitHub Copilot
- Programming languages used
- Files modified with Copilot's assistance
Tracking Methodology
The extension uses a sophisticated heuristic approach to identify code generated by Copilot:
- Copilot Activation Detection: Detects when Copilot is triggered to offer suggestions
- Code Analysis: Analyzes inserted code for patterns typical of AI-generated content
- Timing Analysis: Considers timing between suggestion and insertion
- Content Filtering: Filters out empty lines and comment-only lines for accurate code counting
- Structural Analysis: Requires code to have proper structure (functions, blocks, etc.)
This methodology provides a reasonable approximation of Copilot's contributions, though it may occasionally miss some suggestions or incorrectly attribute manually written code in rare cases.
Time and Cost Savings Calculation
Time savings are calculated using industry benchmarks:
- 30 keystrokes per line of code
- 0.4 minutes saved per Copilot-generated line (based on average typing speed and thinking time)
Financial savings are calculated by:
- Converting time saved to hours: (Lines of code × 0.4) ÷ 60
- Multiplying by your custom developer hourly rate
- Presenting daily, weekly, and monthly financial impact
Billing Savings Calculation
The Billing Savings tab helps you quantify the financial impact of using GitHub Copilot:
- Custom Developer Rate: Enter your hourly development rate
- Projected Savings: View calculated savings based on your actual Copilot usage
- ROI Analysis: See the return on investment for your Copilot subscription
- Cost Comparison: Compare manual coding costs vs. AI-assisted development
- Savings Reports: Download financial impact reports for sharing with management
The cost savings calculation uses the following formula:
Time saved (in hours) = (Lines of code × 0.4) ÷ 60
- Where 0.4 is the estimated time in minutes saved per line
- Divided by 60 to convert minutes to hours
Financial savings = Time saved × Developer hourly rate
- Using your configured hourly rate or the default of $50/hour
For example, if Copilot generates 500 lines of code and your rate is $50/hour:
- Time saved = (500 × 0.4) ÷ 60 = 3.33 hours
- Financial savings = 3.33 × $50 = $166.50
How Copilot Code Detection Works
GitHub Copilot uses advanced machine learning models based on Transformer architecture to detect patterns in code and provide suggestions. Here's how the detection process works:
Pattern Recognition
- Copilot analyzes the surrounding code context, including variable names, function signatures, comments, and adjacent code logic
- It identifies programming patterns and structures that match its training data
- The AI recognizes coding conventions and common implementations for specific tasks
Contextual Understanding
- The tool examines the immediate context (current file) and broader project context
- It considers imports, dependencies, and referenced libraries
- Comments and documentation strings heavily influence the suggestions offered
Training and Knowledge Base
- Trained on billions of lines of public code from GitHub repositories
- Learns from various programming styles, patterns, and implementations
- Continuously improves through feedback mechanisms and model updates
Suggestion Generation
- Generates multiple possible completions ranked by relevance and likelihood
- Filters suggestions based on similarity to existing code in the repository
- Adapts to your coding style as you accept or reject suggestions
Limitations
- May occasionally suggest outdated or suboptimal code patterns
- The confidence of suggestions varies based on the uniqueness of the code context
- Can sometimes struggle with highly complex or unusual code structures
Copilot's detection capability improves over time as it learns from interactions and as the underlying models are enhanced.