🔐 Secret Leak Detector

Secret Leak Detector is a dependable, context-aware credential exposure detection and remediation system for Git repositories and VS Code. It combines multi-signal pattern detection, Shannon entropy analysis, lexical & AST parsing, fail-closed Git commit protection, exposure graph analysis, and transactional remediation to prevent credentials from ever leaking to version control.

📊 Multi-Scanner Empirical Benchmarks (Author-Reported Internal Results)

Notice on Verification Scope: All detection metrics and performance statistics reported below represent internal benchmark results conducted by the authors on our controlled generator and curated real-world corpus using seeded PRNG (0x5eec73) and standalone binaries. While full reproduction scripts are provided (npm run benchmark:multi, npm run benchmark:realworld), these figures must be treated as author-reported findings pending independent third-party replication.

Suite A: 1,000-File Controlled Benchmark Corpus (95% Wilson CIs)

500 True Positives, 400 False Positives, and 100 Adversarial Obfuscations generated deterministically with seeded PRNG (0x5eec73):

Suite B: 100-File Independently Labelled Real-World Corpus

50 Real-World Leaks (GitHub Actions, Dockerfiles, Terraform tfvars, Django settings, Kubernetes manifests, Go APIs, and C# configs) vs 50 Real-World High-Entropy Noise cases (Subresource Integrity SHA digests, 40-char Git commit SHAs, UUIDs, PNG pixels, RFC 7519 JWT examples):

🛡️ Engineering Improvements & Failure Analysis: In prior revisions, real-world recall was 78% due to unescaped @ characters in database passwords disrupting URI boundaries, and unquoted AccountKey=... parameters inside compound Azure connection strings. These patterns were systematically resolved in v1.0.0 with dedicated regression tests, bringing real-world recall to 100% while maintaining 0% false positives on localhost development configs.

Fail-Closed Security Guarantees:

• Git Protection: Scan errors return explicit ERROR status and fail closed (commit blocked when enforcement is enabled).
• Accurate Staged Diff Mapping: Staged diffs are inspected by hunk headers and mapped against the full staged blob, eliminating line offset drift.
• Hook Safety: Pre-commit hooks back up existing hooks to .git/hooks/pre-commit.sld-backup and chain execution.
• Safe Remediation: Environment variable extractions check duplicate keys, escape quotes/newlines, and support rollback.

🌐 Pipeline Separation: Web Showcase vs Extension Pipeline

To ensure absolute engineering hygiene, the project maintains strict separation between web hosting and binary validation:

• Static Web Showcase (Vercel): Hosts the static client-side playground and documentation from public/. It is intentionally decoupled from Node compilation steps to ensure edge CDN reliability without runtime build overhead. A green Vercel deployment confirms web showcase delivery, not extension functionality.
• VS Code Extension & CLI Verification (GitHub Actions CI): The actual extension and CLI engine are authoritatively built and validated via npm run build (node esbuild.js --production), the 99-test security suite (npm test), regression gate (npm run benchmark:gate), and package verification (npm run package producing secret-leak-detector-1.0.0.vsix with both Extension and CLI bundles). CI is the sole source of truth for code correctness.

🎯 The Philosophy: Beyond Simple Regex Scanning

A basic secret scanner answers:

“Does this file contain something that matches an API key regex?”

Secret Leak Detector answers:

“Is this credential actually exposed, how confident are we, what is the explainable evidence, where did it come from in Git history, what is its blast radius, and has remediation proven that exposure is eliminated?”

                  SECRET LEAK DETECTOR PIPELINE
                                │
          ┌─────────────────────┴─────────────────────┐
          │                                           │
       DETECT                                       PROVE
          │                                           │
   Pattern Engine                              Evidence Model
   Entropy Engine                              Multi-Signal Score
   Compiler AST / Lexical                      False-Positive Gate
          │                                           │
          └─────────────────────┬─────────────────────┘
                                │
                              RISK
                                │
                    Dynamic 0-100 Score
                    Blast Radius Engine
                                │
          ┌─────────────────────┴─────────────────────┐
          │                                           │
        TRACE                                        FIX
          │                                           │
   Git Exposure Graph                          Controlled Patch Preview
   Author & Duration                           Approval Gate
   Branch Reachability                         .env & .gitignore Auto-Sync
          │                                           │
          └─────────────────────┬─────────────────────┘
                                │
                             RESCAN
                                │
                    Working-Tree Verification
                    History Purge Verification
                    Delta Reporting (Before vs After)

⚡ Core Capabilities

1. DETECT — Multi-Signal Candidate Engine

Never relies solely on regex patterns. Candidates pass through 5 discrete filters:

1. Provider Signature: 12+ cloud and identity provider patterns (AWS, GitHub, Google Cloud, OpenAI, Anthropic, Stripe, Slack, Microsoft Azure, Database URIs, Cryptographic PEM Keys, JWT).
2. Shannon Entropy Engine: Evaluates information entropy ($H = -\sum p_i \log_2 p_i$) and character set distribution (Base64, Hex, Alphanumeric).
3. Compiler-Backed AST Analysis (TypeScript/TSX): Employs the official TypeScript compiler parser (ts.createSourceFile) to inspect binary expressions (process.env.KEY || "...", ??), nested object property assignments (config.auth.stripeKey), and call expressions (logger.info()). For Python, Go, Shell, and other languages, high-performance structural lexical regex analyzers are used as fallbacks.
4. File Environment Analysis: Adjusts confidence based on file criticality (.env, credentials.json vs docs/, tutorial.md).
5. Adversarial De-Concatenation: Resolves split strings ("ghp_" + "12345...") and comment-interrupted assignments before evaluation.

2. PROVE — Explainable Evidence Model

Every finding produces transparent, auditable evidence rather than a black-box alert:

Finding [#1](https://github.com/MokuLakshithReddy/Secret_Leak_Detector/issues/1): GitHub Personal Access Token (GitHub)
File:        src/config/auth.ts:17:14
Secret:      ghp_••••••••••••••••••••••••••••••••3a9b
Confidence:  99%
Risk Score:  83/100 [CRITICAL]
Blast Radius: SEVERE (Remote & history exposure)

Evidence:
  • [+0] Matched recognized credential signature for GitHub (GitHub Personal Access Token)
  • [+5] Cryptographic entropy: 4.81 bits/char (BASE64 set), indicates non-human randomness
  • [+10] Assigned to sensitive identifier: 'GITHUB_TOKEN'
  • [+8] Located in configuration or secret storage file (.ts)
  • [Passed] Not a documentation placeholder or test mock

3. RISK — Dynamic 0–100 Risk Engine & Model Consistency

Replaces static severity labels with a calibrated formula:

$$\text{Risk Score} = \text{Credential Severity Base} + \text{Confidence Alignment} + \text{Storage Environment} + \text{Exposure Reach}$$

• 0–25: LOW
• 26–50: MEDIUM
• 51–75: HIGH
• 76–100: CRITICAL

Risk Model Consistency & Boundary Validation: Formally verified via automated test suite for mathematical calibration:

• Storage environment penalties: $\text{Score}{\text{config}} > \text{Score}{\text{src}} > \text{Score}_{\text{test}}$
• Reachability monotonicity: $\text{Score}{\text{remote}} > \text{Score}{\text{history}} > \text{Score}_{\text{uncommitted}}$
• Strict $[0, 100]$ boundary constraints across all 12 provider rules. (Note: Validates mathematical and behavioral consistency of the scoring model, rather than claiming statistical prediction of real-world security breaches).

4. TRACE — Git Exposure Graph & Provenance Analysis

Even if a secret is deleted from the current file, Secret Leak Detector traces Git history to derive a Git exposure graph based on credential-related commits and reachable parent relationships:

• Distinguishes direct commit DAG parents (DIRECT_PARENT) from chronological event transitions (EVENT_SEQUENCE).
• Calculates exposure duration in days from introduction to removal.
• Checks contaminated branch reachability (git branch -a --contains) and remote tracking status (origin/*).

$ secret-leak-detector trace <CREDENTIAL_TO_INSPECT>

============================================================
📊 GIT PROVENANCE & BLAST RADIUS TRACE
============================================================
Status:             ACTIVE IN HEAD & COMMITTED
In Active HEAD:     YES 🚨
In Git History:     YES ⚠️ (Rotation mandatory)
Exposure Duration:  18 days
Exposed Branches:   main, origin/main, feature/auth
Remote Pushed:      YES (External exposure risk!)
Introduced By:      Engineer Name <sha: 98d1fe4>
Commit Date:        2026-09-20 23:31:51 +0530
Commit Message:     "feat: implement auth credentials"
============================================================

5. FIX — Controlled Remediation Engine

• Safe Masking: Secrets are masked by default (AKIA••••••••••••DEF) with deterministic SHA-256 fingerprints to prevent secondary log leaks.
• Controlled Patch Preview: Unified diff preview before applying changes.
• Safe .env Migration: Automatically updates .env, generates blank .env.example templates, and secures .gitignore.
• Git Purge Commands: Generates exact git-filter-repo and BFG Repo-Cleaner commands for historical leaks.

6. RESCAN & BASELINE — Enterprise CI/CD Integration

• Verification Engine: Rescans target files to ensure hardcoded secrets are gone, variables resolve correctly, and alerts developers if historical commits still contain the key.
• Baseline System (.secretleak-baseline.json): Records existing known findings so legacy codebases can adopt security guardrails immediately. CI fails only on newly introduced secrets.
• OASIS SARIF v2.1.0 Export: Directly integrates with GitHub Code Scanning alerts.

💻 CLI Usage

# Scan current repository or directory
npx secret-leak-detector scan .

# Scan Git staged changes before commit
npx secret-leak-detector scan --staged

# Output OASIS SARIF v2.1.0 report for CI/CD
npx secret-leak-detector scan . --sarif > results.sarif

# Evaluate with baseline (ignores existing known findings, fails only on new)
npx secret-leak-detector scan . --baseline .secretleak-baseline.json

# Scan full Git commit log for historical secret leaks
npx secret-leak-detector history --max 100

# Trace provenance and blast radius of a credential
npx secret-leak-detector trace <CREDENTIAL_TO_INSPECT>

# Output exposure DAG timeline with reachability & merge awareness
npx secret-leak-detector timeline <CREDENTIAL_TO_INSPECT>

# Export formal Git DAG Exposure Graph (nodes, edges, branch reachability)
npx secret-leak-detector graph <CREDENTIAL_TO_INSPECT> --json

# Interactive controlled remediation (patch preview & .env extraction)
npx secret-leak-detector fix src/config/aws.ts

# Rescan and verify that credential exposure is resolved
npx secret-leak-detector verify src/config/aws.ts

# Install standalone Git pre-commit hook
npx secret-leak-detector hook install

🤖 GitHub Action Integration

Add Secret Leak Detector to .github/workflows/security.yml:

name: Secret Security Gate
on: [push, pull_request]

jobs:
  secret-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run Secret Leak Detector
        uses: MokuLakshithReddy/Secret_Leak_Detector@main
        with:
          path: '.'
          baseline: '.secretleak-baseline.json'
          sarif-output: 'results.sarif'
          fail-on-findings: 'true'

      - name: Upload SARIF to GitHub Security Tab
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: results.sarif

🖥️ VS Code Extension Usage

1. Open any workspace in VS Code.
2. In-editor real-time squiggly markers highlight credential exposures as you type or save.
3. Hover over findings to inspect confidence score, Shannon entropy, and evidence items.
4. Press Alt+. (Windows/Linux) or Cmd+. (Mac) to trigger One-Click Quick Fix:
   • Automatically migrates the credential to .env.
   • Adds .env to .gitignore.
   • Generates .env.example.
   • Replaces literal in code with process.env.<VAR_NAME>.
5. Open the Secret Leak Detector Activity Bar Panel to view active leaks categorized by risk score and jump directly to offending code.
6. Open the Security Dashboard for interactive blast radius and status visualization.

🛡️ Threat Model

Secret Leak Detector is engineered against 6 primary security threats:

• T1: Accidental Working-Tree Commit: Blocked via real-time editor underlines and pre-commit hook diff interception.
• T2: Zombie Git History Exposure: Captured via history commit log scanner, timeline, and graph DAG analysis.
• T3: False Positive Alert Fatigue: Mitigated via Shannon entropy thresholds and placeholder suppression.
• T4: Adversarial Obfuscation Evasion: Neutralized via string de-concatenation, comment stripping, and real compiler AST analysis.
• T5: Secondary Secret Leakage by Scanner: Eliminated through masked displays and SHA-256 fingerprinting.
• T6: Destructive Remediation Regressions: Prevented by diff previews, approval gates, and rescan verification.

For complete threat trees and verification criteria, see docs/threat-model.md.

🧪 Testing & Verification

Run the comprehensive security engineering test suite (77 unit, integration, AST, risk, and regression tests):

npm test

Execute the 1,000-case comparative multi-scanner benchmark (SLD vs Gitleaks vs TruffleHog vs detect-secrets):

npm run benchmark:multi

Execute the 100-case independently labelled real-world benchmark suite:

npm run benchmark:realworld

Execute large-scale performance benchmarking across payloads up to 1GB:

npm run benchmark:scale

Run the automated CI regression gate (enforcing $F1 \ge 99%$, $FPR \le 1%$, $Latency \le 1,000\ \mu\text{s}$):

npm run benchmark:gate

Build the extension and standalone CLI:

npm run build

📜 License

MIT License. Developed for developer-first security engineering.