AI Azure DevOps PR

This extension provides an AI-powered task for Azure DevOps pipelines that integrates with Azure OpenAI to analyze Pull Requests and generate intelligent feedback based on PR diffs and custom prompts.

Features

• Azure OpenAI integration for generating AI-powered responses based on prompts
• Read markdown files as system and user prompts for customizable AI interactions
• Automatic Pull Request diff generation to use as prompt content
• Save AI-generated responses to markdown files
• Post responses as comments directly on Pull Requests
• Easy integration with your existing CI/CD workflows

Requirements

• Azure DevOps Services or Azure DevOps Server 2020 or later
• Azure OpenAI account and API key
• Permissions to comment on Pull Requests (for postToPR functionality)
• Git installed on pipeline agents (for autoDiff functionality)

Installation

From the Marketplace

1. Go to the Azure DevOps Marketplace
2. Search for "AI Azure DevOps PR"
3. Click "Get it free"
4. Select the organization where you want to install the extension

Manual Installation

1. Download the VSIX file from the project releases
2. Navigate to the Extensions section in your Azure DevOps organization (https://dev.azure.com/{organization}/_settings/extensions)
3. Click "Upload a new extension" and select the VSIX file
4. Install the extension in your organization

Usage

Basic Configuration

# azure-pipelines.yml
steps:
- task: AIAzureDevOpsPR@1
  displayName: 'Generate content with Azure OpenAI'
  inputs:
    endpoint: 'https://your-azure-openai-endpoint.cognitiveservices.azure.com/'
    modelName: 'gpt-4'
    deployment: 'your-deployment-name'
    apiKey: '$(AZURE_OPENAI_API_KEY)'  # Use a secret variable
    systemPromptFile: '$(Build.SourcesDirectory)/prompts/system-prompt.md'
    userPromptFile: '$(Build.SourcesDirectory)/prompts/user-prompt.md'
    outputFile: '$(Build.ArtifactStagingDirectory)/openai-response.md'
    postToPR: false
    autoDiff: false

Post Response as Pull Request Comment

# azure-pipelines.yml
trigger:
  - none
pool:
  vmImage: 'ubuntu-latest'

steps:
- checkout: self
  persistCredentials: true  # Important for Git access in autoDiff

- task: AIAzureDevOpsPR@1
  displayName: 'Analyze PR with Azure OpenAI'
  inputs:
    endpoint: 'https://your-azure-openai-endpoint.cognitiveservices.azure.com/'
    modelName: 'gpt-4'
    deployment: 'deployment-name'
    apiKey: '$(AZURE_OPENAI_API_KEY)'
    systemPromptFile: '$(Build.SourcesDirectory)/prompts/code-review-prompt.md'
    userPromptFile: '$(Build.SourcesDirectory)/prompts/additional-context.md'
    outputFile: '$(Build.ArtifactStagingDirectory)/code-review.md'
    autoDiff: true  # Automatically generate PR diff as prompt
    postToPR: true  # Post response as comment on PR

IMPORTANT: To use the postToPR functionality, make sure to enable the "Allow scripts to access OAuth token" option in your pipeline settings.

Sample File Structure

To facilitate setup, you can use this file structure:

repository/
├── .azure-pipelines/
│   └── openai-pr-review.yml  # PR review pipeline
├── prompts/
│   ├── system-prompt.md      # System prompt for OpenAI
│   └── user-prompt.md        # User prompt (optional if using autoDiff)
└── README.md

Input Parameters

Using autoDiff

When autoDiff is set to true, the task:

1. Checks if it's running in the context of a Pull Request
2. Executes the diff generation script between the source and target branches of the PR
3. Uses the generated diff as the user prompt content, instead of the userPromptFile
4. If the diff can't be generated or not in a PR context, falls back to the userPromptFile

This functionality is ideal for automated code reviews, change summaries, or generating documentation based on changes made.

Permissions for PR Comments

To use the postToPR functionality, make sure to:

1. Enable the "Allow scripts to access OAuth token" option in your pipeline settings
2. Have sufficient permissions to comment on PRs
3. Configure the trigger correctly to run in a PR context

Development

Prerequisites

• Node.js 16 or later
• npm 7 or later
• Visual Studio Code (recommended for development)
• Azure DevOps CLI (optional, for publishing)

Project Structure

azure-devops-task/
├── dist/              # Compiled files (generated)
├── images/            # Extension images
├── scripts/           # Utility scripts
├── src/               # Source code
│   ├── index.ts       # Main entry point
│   ├── task.json      # Task definition
│   └── scripts/       # Scripts included with the task
├── tests/             # Tests
├── package.json      # Project configuration
├── tsconfig.json     # TypeScript configuration
└── vss-extension.json # Extension manifest

Available Commands

• Development

  • npm run build: Compile the TypeScript project
  • npm run clean: Remove generated files
  • npm run prepare-package: Prepare files for packaging

• Packaging

  • npm run package: Create VSIX package for distribution
  • npm run package:dev: Create VSIX package with development version

• Publishing

  • npm run publish: Publish extension to the marketplace
  • npm run publish:dev: Publish development version (private)

• Version Management

  • npm run increment-version: Increment project version (patch by default)
  • npm run increment-version -- major: Increment major version
  • npm run increment-version -- minor: Increment minor version

Development Process

1. Clone this repository
2. Run npm install to install dependencies
3. Modify the source code in the src folder
4. Run npm run build to compile and check for errors
5. Run npm run package to create the VSIX package
6. Test the extension in your Azure DevOps environment

Publishing to the Marketplace

To publish the extension to the Azure DevOps Marketplace, you need:

1. A publisher account on the Visual Studio Marketplace
2. Update the publisher field in the vss-extension.json file with your publisher ID
3. Run npm run increment-version to update the version
4. Run npm run publish to package and publish the extension

Local Testing

To test the task locally before publishing:

# Build the project
npm run build

# Copy task.json to the dist folder
cp src/task.json dist/

# Run the test script
node tests/runTest.js

Note: For complete testing with Azure OpenAI, you'll need to provide a valid API key in the test script.

License

ISC

Contributions

Contributions are welcome. Please follow these steps:

1. Fork the repository
2. Create a branch with your feature name (git checkout -b feature/new-feature)
3. Make your changes and commit (git commit -am 'Add new feature')
4. Push to the branch (git push origin feature/new-feature)
5. Create a Pull Request