Elastic PR Reviewer
A VSCode/Cursor extension for Elastic engineers to review pull requests efficiently.
Why?
Elastic has a large influx of new PRs. The introduction of AI assisted coding makes the influx even higher.
Traditionally PRs are reviewed via the Github UI. While this works for some use cases, it could be way better. At the time of writing, Github's UI has been largely stagnant for years. This gives us an opportunity to step in and improve the reviewing experience.
Data needed for a good review
There is not an insignficant amount of data points that an engineer (or an LLM) needs to fully judge the quality of a PR.
A reviewer needs:
- the diffs
- the context in which the diffs are placed (the entire file)
- quick access to the existing files and folders surrounding the diffs
- type information
- information on test coverage
- how the proposed feature behaves functionally, at runtime
Therefore, strictly reviewing from the Github UI means a reviewer does not have enough data to come to a good judgement.
Friction when reviewing Kibana PRs
In addition to this, reviewing PRs at Kibana has friction.
Reviewers tend to look at static code. Some reviewers go further and actually check out the branch and see how the feature behaves.
A reviewer in Kibana needs to:
- find the PR that she needs to review
- switch to the branch
- run yarn kbn bootstrap
- start es + kibana
- optionally load test data / fixtures
- understand the code <-- actual reviewing starts here
- understand the intended behavior
- judge the code in terms of coding standards
- judge the code in context of the product
- judge if the code covers both happy and unhappy paths
This, especially the getting started parts, take effort and are slow (bootstrapping and starting Kibana after switching branches can take somewhere between 1 and 4 minutes).
The speed of bootstrapping / starting Kibana we can't fix easily, but we can improve our tooling to make things easier. If we are succesful in lowering the friction, we can increase the likeliness of reviewers checking out PRs.
This is where this extension comes in.
Features
Team-filtered PR queue — shows only PRs where your team is a requested reviewer
One-click PR checkout — runs gh pr checkout <number> for you
Quickly start ES, Kibana and populate with test data
Analyze the diff using an LLM — provide a summary of every changed files so you quickly build context on what the changed files actually do. Also provide two sorting strategies for the order in which to review: top down and bottom up.
Inline diff view — click any file to open a color-highlighted diff, read and reply to comments by authors and reviewers
Approve, comment or request changes — Get the full discussion inside the IDE
Prerequisites
- GitHub CLI (
gh) installed and authenticated (gh auth login)
- Authenticated against the
elastic org for team detection
Getting Started
Build and install:
cd dev_tools/kibana-pr-reviewer
npm install
npm run compile
# Package as .vsix:
npm run package
# Then: Extensions panel → Install from VSIX...
Open the panel — click the PR Reviewer icon in the Activity Bar (left sidebar).
First run — the extension auto-detects which @elastic/team-* teams you belong to via the GitHub API. This is cached. Override via settings if needed.
Configuration
Open Settings > Kibana PR Reviewer:
| Setting |
Default |
Description |
kibana-pr-reviewer.repo |
elastic/kibana |
GitHub repo to watch |
kibana-pr-reviewer.userTeams |
[] |
Override team detection (e.g. ["@elastic/kibana-core"]) |
kibana-pr-reviewer.llmProvider |
none |
openai or anthropic for LLM file ordering |
kibana-pr-reviewer.llmModel |
(auto) |
Model override |
kibana-pr-reviewer.prLimit |
50 |
Max PRs to fetch |
Setting your LLM API key
Run > Elastic PR Reviewer: Set LLM API Key from the command palette. The key is stored in VS Code's encrypted SecretStorage.
How File Ordering Works
Files are ordered from most important to review first:
- Core implementation — non-test
.ts/.tsx source files
- Entry points & APIs —
index.ts, plugin.ts, route handlers
- Tests —
.test.ts, .spec.ts (review after you understand the change)
- Configuration —
package.json, kibana.jsonc, tsconfig.json
- Documentation —
*.md, *.mdx
With an LLM API key configured, the extension sends the PR title, description, and file list to the model and uses its judgment to refine the ordering, annotating each file with a group label (e.g. "core change", "supporting").
Development
npm install
npm run watch # rebuild on change
# In VS Code: F5 to launch Extension Development Host
