Grader

Current release: 0.10.0

今回の公開・検証範囲は、新課程と共通課題の Rank00〜Rank02、19課題 です。誤FAILの削減と原因表示を更新しました。

• 原因、対象ファイル、修正の方向、参照PDFの版・ページ・要件を日本語で表示します。実行ログは詳細から開けます。
• Python Moduleの出力例との完全一致や、指定されていない内部実装を要求しません。公開API・必要なデータ・明示された動作を確認します。
• get_next_lineは複数BUFFER_SIZE、malloc/read失敗時の後始末、不正アクセスを検査します。追加のStrictストレス方針はPDFの要求と分けて表示します。
• ツール不足、未対応の観測、前提未完了を提出物のFAILに変換しません。必須検査が揃わない結果は「未完了」です。

A-Maze-ingの操作観測は対応するASCIIターミナル表示に限り、MLX画面の操作検証は未対応です。Born2beRootの実機検査には対象VMが必要です。必要な環境や観測方法がない場合は未完了を表示し、手動採点待ちには置き換えません。各OS/CPUのビルドは検証しましたが、ネイティブ実行を実測した環境はLinux x64です。

Rank03以降は開発中で、今回の公開検証範囲には含めません。全29課題の定義と既存の観察器、指定されたCall Me MaybeのMoulinette ZIPは保持していますが、後半課題の完全な動作や機械採点への適合を検証済みとは扱いません。

Python Module 05 bis・06 bisを通常版と別の課題として採点します。python05bis / python06bis で選択でき、通常版の05・06の選択は維持されます。bisは共有パッケージを含む課題全体を採点します。PDFで指定された意図的なエラーを区別し、仮想環境・依存ライブラリのコードをflake8/mypyの対象から除きます。

ex単位の採点と保存時採点（0.8.0）

Graderサイドバーまたはコマンドパレットの exを選んで採点 から、 Pythonプロジェクトの課題定義にあるexを選べます。 選択exの検査だけを実行し、レポートには ex限定採点 と表示します。 全体の合否を確認するときは通常のワークスペース採点を使ってください。

保存時の自動採点設定 では次を変更できます。

{
  "grader.autoGrade.enabled": true,
  "grader.autoGrade.scope": "exercise",
  "grader.autoGrade.delay": 1000,
  "grader.autoGrade.extensions": [".py", ".pyi"],
  "grader.autoGrade.openReport": false
}

初期状態はOFFです。exercise は保存したファイルのexだけ、project は 検出したプロジェクト全体を採点します。連続保存は最後の保存から指定時間待ち、 同じexの実行待ちをまとめます。異なるexの待機分は保持し、engineは順番に実行します。 レポート自動表示がOFFでもサイドバーと診断は更新します。 プロファイル・Norm・メモリ検査などは既存の採点設定を使います。

NOTE: 保存時採点は信頼済みワークスペースでのみ実行します。ex外のファイルや プロジェクトを特定できない保存では、ダイアログを出さず採点を見送ります。 共通ファイル変更の検査には project を選んでください。 ex採点はそのexに割り当てられた検査の評価で、全体のレビュー項目は対象外です。 カスタムengineを設定している場合は、-exercise対応版への更新が必要です。

Grader is a VS Code extension for running ccgrader against 42-style submissions. It is designed for strict local review, not as an official school grading tool.

Version 0.10.0 focuses release validation on the 19 Rank00–Rank02 subjects. The current 29-subject standard catalog retains later-rank definitions for ongoing development; Rank03 and above are outside this release's validation scope. Executed checks report observed PASS or FAIL. Required checks that cannot run because a tool, isolation backend or runtime is unavailable keep the project INCOMPLETE; the report explains the missing capability. An observed submission defect still reports FAIL.

What It Does

• Runs the engine asset matching the workspace host across Linux, macOS and Windows x64/arm64, with existing WSL for Windows POSIX execution and an explicit local engine override.
• Detects the current project from the active editor/workspace so Grade Workspace can run without picking a folder or project.
• Lets you pin a project manually with Grader: Configure Project... when auto-detection picks the wrong target.
• Adds Activity Bar, editor title, and status-bar launchers for grading and quick settings.
• Supports English and Japanese extension UI/report text with grader.language.
• Runs subject-backed Bonus checks only when explicitly enabled, so mandatory-only submissions are not failed for missing optional files.
• Detects Python Module submissions from module folders or individual ex*/ exercise folders.
• Uses a project .venv, venv, or env Python interpreter before falling back to system python3.
• Shows live grading progress from the engine while checks are running.
• Shows projects and behavior checks in the VS Code Testing view.
• Runs one shared engine grade when several Testing-view nodes from the same project are selected.
• Publishes failures to Problems diagnostics when a check maps to a submitted file.
• Opens a readable HTML report for grade, doctor, resource-check, and self-check results.
• Breaks grading results down by the learner-facing unit: Libft functions such as ft_split, Python exN exercises, and project-specific criteria or features. The same item_id hierarchy is available in JSON, CLI output, the Results tree, and HTML reports.
• Keeps PDF-required credential, sandbox and other subject requirements in the selected plan; unavailable observations remain visible instead of being silently excluded.
• Displays observed PASS/FAIL, required incomplete work, actionable Japanese diagnostics, and separate mandatory/Bonus track status.
• Shows a status bar indicator for engine availability, running state, last failure count, and last report.
• Provides a Getting Started walkthrough and view welcome actions for first-run grading.
• Copies the last engine command so failures can be reproduced in a terminal.
• Groups clause-backed results into closed Project setup, Norm and style, Compile and build, Memory and resources, Behavior and tests, Integration and runtime, Security, Documentation, and Other panels. Panels distinguish observed defects from unavailable prerequisites.
• Keeps protected subject resources out of the Marketplace package.

Supported Platforms

The package includes Linux, macOS and Windows engine assets for x64 and arm64. The workspace extension host selects its own OS and CPU, including Remote SSH and WSL hosts. Cross-built assets alone do not establish native runtime guarantees; native CI and runtime capability probes are separate checks.

On Windows, an existing available WSL distribution is selected automatically for POSIX grading, with structured path translation for spaces and Unicode. Without WSL, native static checks remain available; unavailable execution checks are reported as incomplete. Linux namespace and macOS Seatbelt checks run only when their actual host probes succeed. No architecture fallback is used.

Python dependencies are checked for the actual host and ABI. The existing vendored pydantic_core files are Linux x64 only. Other hosts need a trusted system Python with compatible dependencies. Missing tools or incompatible ABIs produce a dependency diagnostic rather than a submission failure. Other native OS/CPU combinations have a CI workflow contract, but were not executed on this Linux development host.

Setup

1. Open a submission workspace or any file inside a submission.
2. Click the Grader icon in the Activity Bar, editor title, or status bar, then run Grade Current Project.
3. On the workspace execution host, the extension detects the current project and uses the bundled ccgrader engine when available.
4. Run Grader: Setup only if you need a private/local resource root or a configured custom engine.
5. Turn on Bonus Checks from the Grader Activity Bar only when the submitted project includes the subject bonus part.
6. Select the cumulative Minimum, Normal, or Strict profile. Extension grading runs Deep and reports execution coverage for each selected criterion.
7. Run Grader: Doctor to verify tools such as cc, make, norminette, valgrind, docker, python3, and uv.

The extension does not assume any local absolute path. Windows uses its matching native asset for static checks and an existing WSL backend for POSIX execution.

Workspace Trust is respected. When a workspace is untrusted, the extension allows specification browsing but does not execute the local grading engine.

Resource Policy

The VSIX includes extension code, platform engines, public specification metadata, trusted runtime dependencies, and the exact pinned Call Me Maybe Moulinette archive recorded in NOTICE.md.

It does not bundle:

• subject PDFs
• official checker binaries
• third-party tester repositories other than the expressly bundled Moulinette asset
• other resource archives such as .zip, .tar, .tgz, .gz, or .whl; the single Moulinette ZIP exception is fixed by path and SHA-256

Configure grader.resourceRoot to point at a private/local resource tree when you have the right to use those resources locally.

Profiles and Accuracy Evidence

Minimum, Normal, and Strict are cumulative: Normal includes Minimum criteria, and Strict includes both earlier levels. Deep attempts every selected criterion. Required unexecuted criteria keep an otherwise passing project INCOMPLETE. Fast defers expensive checks and remains provisional editing feedback.

The release build measures Fast mode over a fixed three-subject frontier for 30 engine runs and rejects the package when p95 exceeds one second. This is a responsiveness budget for a provisional Fast report, not a claim that every Deep or environment-dependent check finishes in one second.

A check receives PASS or FAIL only after its declared observation executes. Missing runners, prerequisites or trusted execution capabilities are unavailable results, with no synthetic PASS or submission FAIL. Mandatory and Bonus tracks are reported separately, and Bonus stays opt-in.

The project aims for at least 99% agreement on its labeled regression corpus, but it does not turn that engineering target into an unsupported probability for an arbitrary submission. Reports expose the actual denominator: required, executed, passed, failed, pending, blocked, unavailable, Fast-deferred, and excluded checks.

The catalog does not publish guessed per-project percentages or a resource-pack precision target. Public Marketplace builds exclude PDFs, official checker binaries and reference answers. The sole archive exception is the exact pinned Call Me Maybe Moulinette ZIP; grader.resourceRoot can enable explicitly identified local evidence without turning unavailable checks into PASS.

Deep C projects such as ft_printf, libft, get_next_line, and push_swap have the strongest automated coverage. Projects that require a VM, browser, live containers, LLM resources, or interactive defense show the unavailable automatic evidence and their reduced coverage explicitly.

Bonus checks are opt-in. With grader.enableBonus off, bonus-only deliverables and fixtures appear as SKIP with bonus.disabled. With it on, the engine passes -bonus and grades subject-specific bonus behavior such as ft_printf width/precision/flags, get_next_line bonus multi-fd/static-variable rules, and the push_swap checker.

Python dependency checks separate required library declarations from the installed runtime. A missing required declaration can fail the submission check; an unavailable compatible runtime is reported with setup hints and leaves the affected required checks incomplete.

Commands

• Grader: Setup
• Grader: Quick Actions
• Grader: Grade Workspace
• Grader: Grade Folder...
• Grader: Doctor
• Grader: Resource Check
• Grader: Self Check Materials
• Grader: Open Specification Catalog
• Grader: Configure Engine...
• Grader: Configure Project...
• Grader: Configure Profile...
• Grader: Configure Mode...
• Grader: Configure Parallelism...
• Grader: Toggle Bonus Checks
• Grader: Open Last Report
• Grader: Copy Last Engine Command
• Grader: Show Output

Configuration

• grader.enginePath: optional override path to a local grading engine
• grader.resourceRoot: path to private/local subject resources
• grader.defaultProject: project selection override. Use auto, ask, all, or a concrete project id
• grader.profile: cumulative minimum, normal, or strict criteria
• grader.mode: fixed to deep for complete automated review
• grader.enableNorm: run Norm checks when available
• grader.enableMemcheck: run valgrind checks when available
• grader.enableBonus: pass -bonus to the engine and run optional subject bonus checks
• grader.parallelism: maximum project workers for multi-project grading; 0 lets the engine choose
• grader.autoOpenReport: open the HTML report after grading
• grader.runDoctorAfterSetup: run Doctor after setup in trusted workspaces
• grader.language: ja, en, or auto
• grader.extraArgs: additional engine arguments

Development

npm run build:engines
npm test
npm run package:vsix

scripts/build-engines.sh cross-builds all six OS/CPU assets and runs smoke checks on its native host. Executable-format checks verify each packaged architecture. Native Linux x64 was measured here; the other five hosts have separate native CI jobs and remain unmeasured until those jobs run.

On older 42 environments, global @vscode/vsce may require a newer Node version. If npm config get prefix points to /usr or /usr/local, set a user prefix first:

mkdir -p ~/.local
npm config set prefix ~/.local
npm install -g @vscode/vsce

If the host Node is too old for the current @vscode/vsce, use a newer Node runtime for publishing.

Publishing

The ignored .env.publish.local may provide the Marketplace token without copying it into the extension:

PUBLISHER=harnakam
VSCE_PAT=

The publish script validates the exact 0.10.0 VSIX and its structured fresh-install record before reading a token file. Keep the token in the ignored .env.publish.local and pass its absolute path as PUBLISH_ENV_FILE.

Use npm run publish:dry-run before publishing. It builds and validates the VSIX once and prints its exact digest without reading credentials. Fresh-install that same file, create the structured smoke record, then use npm run publish:marketplace to revalidate and publish only those exact bytes.

See PUBLISHING.md for the longer runbook.

Disclaimer

This project is an independent local review tool. Missing required tools or execution capabilities are reported as INCOMPLETE with their actual coverage. Always compare against the current subject and official evaluation process.