Universal AI PR Reviewer

An Azure DevOps extension that provides automated AI code reviews for pull requests using any model provider, plus a unified cross-project PR dashboard.

Features

AI Code Review

• 7 AI providers: OpenAI, Azure OpenAI, Azure AI Foundry, Anthropic Claude, OpenAI-compatible (DeepSeek, vLLM, GLM), Custom REST
• Structured reviews: JSON output with confidence scoring, severity levels, and fix suggestions
• 8 review categories: Bugs, Security, Performance, Best Practices, Error Handling, Compatibility, Readability, Secrets
• Multi-language: Reviews any programming language
• PR metadata context: Uses PR title/description for smarter reviews
• Comment management: Auto-cleanup of previous reviews, per-file threads, summary overview

PR Dashboard

• Cross-project: View PRs across all projects in one place
• Three tabs: Needs My Review, Created by Me, All PRs
• AI review status: See review results directly on the dashboard
• Visual indicators: Vote dots, age badges, draft flags, stale PR detection
• Project filtering: Filter by specific project or view all
• Modern UI: Clean, responsive design matching Azure DevOps design language

Quick Start

trigger:
  - main

pr:
  - main

pool:
  vmImage: ubuntu-latest

steps:
  - checkout: self
    fetchDepth: 0

  - task: UniversalAiPrReviewer@1
    inputs:
      providerType: 'openai'
      apiKey: '$(OPENAI_API_KEY)'
      model: 'gpt-4.1-mini'
      reviewFocus: 'bugs,security,performance,bestPractices,errorHandling'
      confidenceThreshold: '75'
    env:
      SYSTEM_ACCESSTOKEN: $(System.AccessToken)

DeepSeek Setup

  - task: UniversalAiPrReviewer@1
    inputs:
      providerType: 'openAICompatible'
      apiKey: '$(DEEPSEEK_API_KEY)'
      customEndpoint: 'https://api.deepseek.com/v1'
      customModelName: 'deepseek-chat-v4'

Custom Model Setup

Use the Custom Model Name field to use any model not listed in the dropdown. Examples:

• deepseek-chat-v4-pro
• gpt-4o-2024-08-06
• my-fine-tuned-model
• claude-3-5-haiku-20241022

Preview

Open these files in your browser to see the UI:

• previews/task-config.html - Pipeline task configuration form
• previews/pr-comments.html - AI review comments on a PR
• previews/dashboard.html - Cross-project PR dashboard

Architecture

├── vss-extension.json           # Extension manifest (task + hub)
├── task.json                    # Pipeline task definition (19 inputs)
├── package.json                 # Dependencies (openai, anthropic, task-lib)
├── tsconfig.json                # TypeScript strict config
├── src/                         # Pipeline task source (TypeScript)
│   ├── index.ts                 # Main orchestrator
│   ├── types/index.ts           # Interfaces, enums, constants
│   ├── utils/
│   │   ├── config.ts            # Input parsing, prompt builder
│   │   └── prContext.ts         # Azure DevOps PR context
│   └── services/
│       ├── aiProvider.ts        # 7 provider implementations
│       ├── gitService.ts        # Git diff & file filtering
│       ├── prCommentService.ts  # Comment CRUD operations
│       └── reviewEngine.ts      # Response parsing & dedup
├── dashboard/                   # PR Dashboard hub (React + Fluent UI)
│   ├── src/
│   │   ├── App.tsx
│   │   ├── main.tsx
│   │   ├── components/
│   │   │   ├── PRDashboard.tsx  # Main dashboard layout
│   │   │   ├── PRCard.tsx       # Individual PR card
│   │   │   └── AIReviewBadge.tsx # AI review status indicator
│   │   ├── services/
│   │   │   └── azureDevOps.ts   # Azure DevOps REST API calls
│   │   ├── types/index.ts
│   │   └── utils/helpers.ts
│   ├── index.html
│   └── index.css
├── previews/                    # Static HTML design previews
│   ├── task-config.html
│   ├── pr-comments.html
│   └── dashboard.html
├── tests/                       # Unit tests (mocha + mock-task)
├── images/                      # Extension icons
├── sample-pipeline.yml          # 8 ready-to-use YAML examples
└── azure-pipelines.yml          # CI/CD for extension publishing

Requirements

• Azure DevOps Services or Azure DevOps Server 2022+
• Agent version 2.209.0+ (supports Node 24/20/16)
• Pipeline: fetchDepth: 0 on checkout, "Allow scripts to access OAuth token" enabled
• Build service: "Contribute to pull requests" permission on the repository