Skip to content
| Marketplace
Sign in
Visual Studio Code>Education>AutoJudge Code RunnerNew to Visual Studio Code? Get it now.
AutoJudge Code Runner

AutoJudge Code Runner

AutoJudge

|
71 installs
| (0) | Free
Run and test your C, C++, Java, JavaScript, PHP, and Python solutions on AutoJudge from VS Code, using local .in and .out test case files. No local compilers needed.
Installation
Launch VS Code Quick Open (Ctrl+P), paste the following command, and press enter.
Copied to clipboard
More Info

AutoJudge Code Runner

Run your solution against your own test cases from VS Code, on the AutoJudge judge. Write .in and .out files next to your code, press one button, and see each case's output or verdict in the editor. You don't need compilers or interpreters on your machine.

Testing solution.py: two cases pass and one shows Wrong Answer with the expected and received output and the first differing line

Features

  • Run or test in one click. Run shows what your program prints for every .in file. Test compares the output with the matching .out file and gives a verdict for each case.
  • Results you can scan. Each case starts with ✓ or ✗, its file name, its verdict, and its run time, in aligned columns. Passes show in green and failures in red in the default themes. Wrong answers show the expected and received output one above the other, with the first line that differs.
  • Runtime errors and time limits are named. A crash shows the error message the judge returned, indented under its case like your program's output, and a summary line counts the cases that failed.
  • Uses your test files. Every .in file next to your code (or in a folder you choose) is a test case. Cases run in natural order, so case2.in runs before case10.in.
  • Catches mistakes before sending. Test mode stops if a .out file is missing and names it. File names the judge would reject are caught first, with a suggested new name. Only one run happens at a time.
  • Works with your own server. Point the extension at a self-hosted AutoJudge server.

[!TIP] Competing in an AutoJudge contest? AutoJudge Contest lets you read problems, export their public test cases as .in / .out files, and submit from VS Code. Use this extension to test against those files before you submit.

Supported languages

Language Extension
C .c
C++ .cpp
Java .java
JavaScript .js
PHP .php
Python .py

Install

Search for AutoJudge Code Runner in the Extensions view (Ctrl+Shift+X / Cmd+Shift+X) and click Install, or run:

code --install-extension autojudge.autojudge-extension

Quick Start

  1. Put your solution and its test cases in one folder. Each .in file is one input; the .out file with the same name is its expected output.

    two-sum/
    ├── solution.py
    ├── case1.in
    ├── case1.out
    ├── case2.in
    └── case2.out
    
  2. Open solution.py and click Run (play icon) or Test (beaker icon) in the editor title bar.

    The Run (play) and Test (beaker) buttons in the editor title bar, with the Run tooltip

  3. Read the result in the AutoJudge output panel. What your program printed is indented under each case, and the status bar shows a short summary for a few seconds. Click the folder path (Cmd+click / Ctrl+click) to show it in the Explorer.

    Run mode output: two cases print their result under the .in file name, the third shows Time Limit Exceeded in red, and the summary line

Unsaved changes are included in the run. If the folder has no .in files, Run runs your program once with empty input.

Commands

Command Command ID What it does
AutoJudge: Run Active File autojudge.runActiveFile Runs the file with each .in file as input and shows what it prints. .out files are not used.
AutoJudge: Test Active File autojudge.testActiveFile Runs the file with each .in file and compares the output with the matching .out file. Every .in file needs a .out file.

Both commands appear in the editor title bar and the Command Palette for .c, .cpp, .java, .js, .php, and .py files saved on disk. One run happens at a time; cancel a run from its progress notification.

Keyboard Shortcuts

Command Windows / Linux macOS
AutoJudge: Run Active File Ctrl+Alt+R Ctrl+Option+R
AutoJudge: Test Active File Ctrl+Alt+T Ctrl+Option+T

Configuration

Setting Default Description
autojudge.baseUrl https://api.autojudge.io AutoJudge API address. Change it only if you use your own server. Base paths such as https://example.com/api are supported.
autojudge.testcasePath "" Folder with your test cases. Leave empty to use the folder of the file you run. Supports ${workspaceFolder}, ${fileDirname}, and other file variables. Relative paths start from the folder of the file you run.
{
  "autojudge.testcasePath": "${fileDirname}/tests"
}

Only .in and .out files directly inside the folder are used; subfolders are ignored. A testcasePath that does not exist or is not a folder stops the run with an error and an Open Settings button.

Troubleshooting

"Test mode needs a .out file for every .in file. Missing …" — Add the missing .out file next to its .in file, or click Run Instead to run without comparing outputs.

"Test mode compares each .in file with its .out file. No .in files found in …" — Add .in / .out pairs to that folder or set autojudge.testcasePath, or click Run Instead.

"Rename … to run it on AutoJudge" — The judge only accepts file names made of letters, digits, _ and -, such as two_sum.py. The extension also asks for a lowercase extension, because older judges hang on names like Main.PY.

"Could not reach the AutoJudge server …" or "did not answer within 20 seconds" — Check your connection and autojudge.baseUrl (click Open Settings). It must be the API address, which is not always the same as the website address.

"The outputs differ only in whitespace" — The judge ignores leading and trailing whitespace but compares spaces and line breaks inside the output. Check for extra spaces between values or extra blank lines. Windows line endings in .in and .out files are converted for you.

"Run failed" followed by "The judge returned results for 2 of 3 cases." — The judge stopped partway through, usually because the whole run took too long. Try again, or run fewer cases at once.

Release Notes

See the CHANGELOG.

Contributing

Issues and pull requests are welcome on GitHub. Development runs in Docker Compose:

docker compose up -d --build
docker compose exec extension npm install
docker compose exec extension npm test

Press F5 in VS Code to launch the Extension Development Host.

License

MIT

  • Contact us
  • Jobs
  • Privacy
  • Manage cookies
  • Terms of use
  • Trademarks
  • Your Privacy Choices
  • Consumer Health Privacy
© 2026 Microsoft