TraceMLRead-only, filesystem-only viewer for ML training runs (tables, plots, comparison) inside VS Code. Works over Remote-SSH. Installation Launch VS Code Quick Open ( Ctrl+P), paste the following command, and press enter.Copied to clipboard |
TraceML
TraceML is a VS Code extension for browsing ML training runs. It finds runs on the filesystem and shows them in a panel inside VS Code: a run table with filtering and sorting, metric plots, a detail view for each run, and side-by-side comparison. The filesystem is the only source of truth. There is no server, no database, no HTTP endpoint and no browser app. Everything runs inside VS Code, including over Remote-SSH, where the extension reads the runs directly on the remote machine. TraceML is read-only for your runs. It never creates, modifies or deletes anything in your run folders (even Delete only hides a run in TraceML), it makes no network calls, and it reads only inside the runs folders you configure. Its own small files (remembered view, run database, CSV exports, saved plots) go to Quick start
How to log for TraceML
What TraceML needs: every training run writes one folder of small text files. TraceML (a VS Code viewer) only reads these files. There is no server, no account and no package to install. Step 1: one folder per runCreate a new folder for every run, inside
Step 2: know the three kinds of values
The run's name and status go into Step 3: the files
At the end: Step 4: follow these rules
Step 5: copy this logger (Python, standard library only)Use it in the training script like this:
To check the result, open the project in VS Code and click the TraceML icon. The run shows up in the table: config columns are tagged InstallationTraceML is shipped as a
The packaged extension doesn't need Node, npm or any other process running. Commands
The run list in the activity barThe TraceML view in the activity bar lists your runs by status, so you can see what's training and what crashed while you edit code:
Settings
The UITraceML uses a dark-only application shell with its own design system, inside a single VS Code webview. It has a sidebar, a header and three views.
Design system. All colors, radii, spacing and type sizes are CSS tokens in
Writing runs: the TraceML run formatThis section is the authoritative specification for producing runs that TraceML can display. It's written so that a person, or an AI coding agent asked to "add TraceML logging to this training script", can implement it exactly. Everything here is 1. Directory layout
2. Write rules (important for live viewing)TraceML reads these files while training is still writing them. Follow these rules and the viewer never sees broken data:
3.
|
| Field | Type | Rules |
|---|---|---|
schema_version |
int | 1. |
id |
string | Same as the directory name. |
name |
string | Short human name, shown in bold in the table. |
group |
string or null | Free-form group (e.g. a sweep name). It can be used for grouping. |
tags |
string[] | Free-form labels. |
notes |
string | Free text. |
status |
string | running → completed, failed or killed. Never write stale; TraceML derives it. |
started_at / ended_at / heartbeat_at |
ISO-8601 UTC or null | ended_at is null while the run is running. |
duration_s |
number or null | Set when the run finishes. |
exit |
object or null | null while running. On finish it's {"code": 0, "signal": null}. For a failure, add "error_type", "error_message" and "traceback" (full text; TraceML shows the tail). |
command, cwd, host, git |
optional | Shown in the run inspector. host and git may contain any keys. |
Status lifecycle: create the file with running → update heartbeat_at periodically → write the final status once. After a crash without cleanup, the run stays running, and TraceML shows it as stale.
4. params.yaml
A flat mapping with dot-separated keys and scalar values: numbers, strings, booleans or null. Every key becomes a table column that can be sorted, filtered and grouped on, and that appears in the comparison view.
data.batch_size: 64
model.hidden: 128
optim.lr: 0.0003
optim.name: "adamw"
Nested YAML is accepted and flattened (optim: {lr: …} → optim.lr). Lists are shown as compact JSON strings. Use the same key names across runs of a project, otherwise the columns don't line up.
5. config.json
The full nested training configuration, as any JSON value. It's shown as a collapsible tree in the run inspector. This file is optional; params.yaml is what powers the table.
6. metrics.jsonl: the metric history
One JSON object per line, appended as training progresses:
{"step": 9, "epoch": 9, "time": 1790000000.123, "train/loss": 0.41, "lr": 0.00021}
{"step": 9, "epoch": 9, "time": 1790000004.551, "val/loss": 0.38, "val/auc": 0.87}
step(int, required): the epoch-level step and the primary x-axis. In TraceML V1,stepnormally equalsepoch. Don't log per batch; log once or twice per epoch.epoch(int): the epoch number.time(float): Unix seconds.- Every other key is a metric. Values must be numbers or
null. A null is drawn as a gap. Don't write strings, lists or objects as metric values. - Name metrics with a prefix so the metric picker can group them:
train/…,val/…,test/…,sys/…(e.g.sys/gpu_mem_gb);lror other names without a prefix go under other. - Rows may be sparse. Log train and val metrics in separate rows with the same
step. - Never write NaN or Infinity. Write
nullinstead. TraceML tolerates Python'sNaNtokens, but other JSON readers don't. - Partial or corrupt lines: a half-written final line is ignored until it's completed, and other corrupt lines are skipped and reported as a warning.
7. metrics.json: the derived summary
Rewrite it atomically after every logged row (or at least every epoch). The run table is built from this file, so TraceML never has to read the full history just to list runs.
{
"schema_version": 1,
"updated_at": "2026-09-29T14:10:00Z",
"step": 19,
"epoch": 19,
"last": {"train/loss": 0.21, "val/auc": 0.88, "lr": 0.000001},
"best": {"val/auc": {"value": 0.894, "step": 14, "epoch": 14, "mode": "max"}},
"summary": {"test/auc": 0.881}
}
step/epoch: the latest logged step.last: the most recently logged value of every metric inmetrics.jsonl(the value may be null).best: for each metric you care about, the best value so far, plus thestepandepochwhere it happened, andmode. The mode ismaxfor AUC or accuracy andminfor losses.summary: final or evaluation values that have no history, e.g.test/*after training. Numbers or strings.- Default columns: the table shows every
bestandsummarymetric as a column by default. The scatter and box plots can use anylast,bestorsummarynumber.
8. Checklist for an implementation
- [ ] A unique run directory
<runs folder>/<YYYYMMDD-HHMMSS>_<name>_<hex4>/is created before training starts. - [ ]
run.json(statusrunning),params.yaml,config.jsonand an emptymetrics.jsonare written atomically at start. - [ ] Each evaluation appends complete lines to
metrics.jsonlwithstep,epoch,timeand prefixed metric names, then flushes. - [ ]
metrics.json(last,best,step,epoch) is rewritten atomically after each log call. - [ ]
heartbeat_atis refreshed at least every 30 s, including during long epochs (use a background thread). - [ ] On exit (normal, exception or Ctrl+C),
status,ended_at,duration_sandexitare written, including the error type, message and traceback on failure.summarygets the test metrics. - [ ] No NaN/Infinity, no per-batch rows, no in-place rewrites of
metrics.jsonl.
9. Reference logger (Python standard library only)
This is a complete, tested implementation of the rules above. The same file is in the repository as scripts/traceml_logger.py. Copy it into your project, or give it to an agent as the specification.
"""Minimal reference writer for the TraceML run format (schema_version 1). Python stdlib only.
Copy this file into a training project (or use it as the specification for your own logger).
from traceml_logger import TraceMLRun
with TraceMLRun("runs", "baseline", params={"optim": {"lr": 3e-4}, "model.hidden": 128},
group="lr-sweep", tags=["cv"], best={"val/auc": "max", "val/loss": "min"}) as run:
for epoch in range(num_epochs):
run.log(epoch, {"train/loss": train_loss, "lr": lr})
run.log(epoch, {"val/loss": val_loss, "val/auc": val_auc})
run.summary["test/auc"] = test_auc
Leaving the `with` block normally marks the run `completed`; an exception marks it `failed`
(with error type, message and traceback); Ctrl+C marks it `killed`.
"""
from __future__ import annotations
import json
import math
import os
import secrets
import socket
import sys
import threading
import time
import traceback
from datetime import datetime, timezone
SCHEMA_VERSION = 1
def _iso(t: float) -> str:
return datetime.fromtimestamp(t, timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
def _atomic_write(path: str, text: str) -> None:
"""Write to a dot-prefixed temp file, then rename: readers never see a half-written file."""
tmp = os.path.join(os.path.dirname(path), f".{os.path.basename(path)}.tmp")
with open(tmp, "w", encoding="utf-8", newline="\n") as f:
f.write(text)
f.flush()
os.fsync(f.fileno())
os.replace(tmp, path)
def _yaml_scalar(v) -> str:
if v is None:
return "null"
if isinstance(v, bool):
return "true" if v else "false"
if isinstance(v, (int, float)):
return repr(v)
return json.dumps(str(v))
def _flatten(d: dict, prefix: str = "") -> dict:
out: dict = {}
for k, v in d.items():
key = f"{prefix}{k}"
if isinstance(v, dict):
out.update(_flatten(v, key + "."))
elif isinstance(v, (list, tuple)):
out[key] = json.dumps(list(v))
else:
out[key] = v
return out
def _metric(v):
"""Numbers become floats; None / NaN / inf become null (a gap in the plot)."""
if v is None:
return None
v = float(v)
return v if math.isfinite(v) else None
class TraceMLRun:
def __init__(self, runs_dir: str, name: str, params: dict | None = None, config: dict | None = None,
group: str | None = None, tags=(), notes: str = "", best: dict | None = None,
heartbeat_s: float = 15.0):
self.t0 = time.time()
stamp = datetime.fromtimestamp(self.t0, timezone.utc).strftime("%Y%m%d-%H%M%S")
self.id = f"{stamp}_{name}_{secrets.token_hex(2)}"
self.dir = os.path.join(runs_dir, self.id)
for sub in ("logs", "checkpoints", "artifacts"):
os.makedirs(os.path.join(self.dir, sub), exist_ok=True)
self.best_modes = dict(best or {}) # metric -> "max" | "min"
self.last: dict = {}
self.best: dict = {}
self.summary: dict = {}
self.step = None
self.epoch = None
self._lock = threading.Lock()
self.meta = {
"schema_version": SCHEMA_VERSION,
"id": self.id,
"name": name,
"group": group,
"tags": list(tags),
"notes": notes,
"status": "running",
"started_at": _iso(self.t0),
"ended_at": None,
"heartbeat_at": _iso(self.t0),
"duration_s": None,
"exit": None,
"command": [sys.executable, *sys.argv],
"cwd": os.getcwd(),
"host": {"hostname": socket.gethostname(), "pid": os.getpid(), "python": sys.version.split()[0]},
}
flat = _flatten(params or {})
_atomic_write(os.path.join(self.dir, "params.yaml"), "".join(f"{k}: {_yaml_scalar(v)}\n" for k, v in flat.items()))
_atomic_write(os.path.join(self.dir, "config.json"), json.dumps(config if config is not None else (params or {}), indent=2, default=str) + "\n")
self._jsonl = open(os.path.join(self.dir, "metrics.jsonl"), "a", encoding="utf-8", newline="\n")
self._write_meta()
self._write_metrics_json()
self._stop = threading.Event()
threading.Thread(target=self._heartbeat, args=(heartbeat_s,), daemon=True).start()
# -- writing ---------------------------------------------------------------
def log(self, step: int, metrics: dict, epoch: int | None = None) -> None:
"""Append one row to metrics.jsonl (step = epoch-level step) and refresh metrics.json + heartbeat."""
row = {"step": int(step), "epoch": int(step if epoch is None else epoch), "time": round(time.time(), 3)}
for k, v in metrics.items():
row[k] = _metric(v)
with self._lock:
self._jsonl.write(json.dumps(row) + "\n") # one complete line per write
self._jsonl.flush()
self.step, self.epoch = row["step"], row["epoch"]
for k, v in row.items():
if k in ("step", "epoch", "time"):
continue
self.last[k] = v
mode = self.best_modes.get(k)
if mode and v is not None:
b = self.best.get(k)
if b is None or (v > b["value"] if mode == "max" else v < b["value"]):
self.best[k] = {"value": v, "step": row["step"], "epoch": row["epoch"], "mode": mode}
self._write_metrics_json()
self.meta["heartbeat_at"] = _iso(time.time())
self._write_meta()
def finish(self, status: str = "completed", exc: BaseException | None = None) -> None:
self._stop.set()
now = time.time()
with self._lock:
if not self._jsonl.closed:
self._jsonl.close()
exit_info: dict = {"code": 0 if status == "completed" else 1, "signal": "SIGINT" if status == "killed" else None}
if exc is not None and status == "failed":
exit_info.update(
error_type=type(exc).__name__,
error_message=str(exc),
traceback="".join(traceback.format_exception(type(exc), exc, exc.__traceback__)),
)
self.summary = {k: (_metric(v) if isinstance(v, (int, float)) or v is None else v) for k, v in self.summary.items()}
self._write_metrics_json()
self.meta.update(status=status, ended_at=_iso(now), heartbeat_at=_iso(now), duration_s=round(now - self.t0, 3), exit=exit_info)
self._write_meta()
def __enter__(self) -> "TraceMLRun":
return self
def __exit__(self, exc_type, exc, tb) -> bool:
if exc_type is None:
self.finish("completed")
elif issubclass(exc_type, KeyboardInterrupt):
self.finish("killed", exc)
else:
self.finish("failed", exc)
return False # never swallow the exception
# -- internals -------------------------------------------------------------
def _write_meta(self) -> None:
_atomic_write(os.path.join(self.dir, "run.json"), json.dumps(self.meta, indent=2) + "\n")
def _write_metrics_json(self) -> None:
_atomic_write(
os.path.join(self.dir, "metrics.json"),
json.dumps({
"schema_version": SCHEMA_VERSION,
"updated_at": _iso(time.time()),
"step": self.step,
"epoch": self.epoch,
"last": self.last,
"best": self.best,
"summary": self.summary,
}, indent=2) + "\n",
)
def _heartbeat(self, every: float) -> None:
# Keeps heartbeat_at fresh during long epochs so the run is not shown as stale.
while not self._stop.wait(every):
with self._lock:
if self._stop.is_set():
return
self.meta["heartbeat_at"] = _iso(time.time())
self._write_meta()
Status and stale
stale is derived by TraceML; it's never written to disk. A run is stale when status == "running" and now - heartbeat_at > traceml.staleAfterSeconds. A completed, failed or killed run is never stale. If a running run has no heartbeat_at, it's shown as running?, because its staleness can't be judged. TraceML never invents a heartbeat time.
Live refresh
- TraceML polls every
refreshIntervalSeconds, and only while the panel is open. Each poll works like this:- It lists the configured runs folders (one directory read each), so new runs appear and deleted runs disappear. Runs deleted from TraceML are skipped.
- It stats the core files of each running run and re-parses only files whose mtime or size changed.
- It advances the
metrics.jsonlof the plotted runs by reading only the newly appended bytes. If the file shrank, it starts over from the beginning.
- Finished runs are frozen. Completed, failed and killed runs aren't re-checked on polls, because their files no longer change. They're checked again (by stat, re-parsed only if changed) when you press Refresh, open the panel or change a setting.
- Run database. The runs are saved in
traceml/.cache/runs.sqlite(at most every 10 s while runs change; only changed runs are rewritten). When TraceML opens, the table is filled from it right away. Then each run's files are stat'ed once, and only changed files are parsed again. Without the database, the first scan parses every run. It's only a cache: deleting it is harmless, and a corrupt, edited or other-version file is rebuilt from the run folders. Aruns-index.jsonleft by 0.1.17 is read once and migrated. See The run database. - Only changed runs are sent to the panel. Sort order, filters, selection, zoom and column choices are all kept across updates.
- Stale runs change status automatically as their heartbeat ages.
- Polling stays the reliable baseline even if a filesystem watcher is added later.
Remote-SSH
With "extensionKind": ["workspace"], the extension host, and so all filesystem access, runs on the remote machine next to the runs. Only small JSON messages cross the connection: run summaries, and downsampled series for the runs you're plotting. Nothing needs to be installed on the remote beyond the extension itself. You don't need Node, npm or Python packages there. To browse runs outside the opened folder, add an absolute path to traceml.roots, e.g. "/scratch/me/project/runs".
The run database
traceml/.cache/runs.sqlite is a small SQLite file with the runs TraceML shows (deleted runs excluded). Open it with sqlite3, DB Browser for SQLite, pandas (pd.read_sql) or give it to an agent:
| Table | Columns |
|---|---|
runs |
key (run directory), id, name, group_name, tags (;-separated), notes, status (from run.json), display_status (adds stale), started_at / ended_at / heartbeat_at (ISO-8601 UTC), duration_s, step, epoch, path, location, legacy, warnings |
config |
run_key, key, value: one row per config value (params.yaml); booleans are 1/0 |
metrics |
run_key, metric, kind (start, last, best or summary), value, and for best the step and mode |
-- the 5 runs with the best val/auc, with their learning rate
SELECT r.name, c.value AS lr, m.value AS best_auc
FROM runs r
JOIN metrics m ON m.run_key = r.key AND m.metric = 'val/auc' AND m.kind = 'best'
LEFT JOIN config c ON c.run_key = r.key AND c.key = 'optim.lr'
ORDER BY best_auc DESC LIMIT 5;
- The full per-epoch history is not copied; it stays in each run's
metrics.jsonl(pathpoints to the run folder).run_cacheandmetaare TraceML's own bookkeeping. - The file is replaced atomically, so a reader never sees half a file. It reflects the runs as of the last save (up to 10 s old); treat it as read-only, since TraceML overwrites it.
- SQLite runs inside the extension as WebAssembly (sql.js, bundled). There is no native module, no server and nothing to install, also over Remote-SSH. If it can't be loaded, TraceML falls back to
runs-index.json.
Exporting the table for an agent
Export (in the runs toolbar) writes the rows currently shown in the table to a CSV file: filters, sort order, column order and Max rows applied, with the visible columns only. A bar then shows the file path, with Copy path and Open (read-only). Give that path to an agent (e.g. "read this CSV and compare the runs"), or load it with pandas.read_csv.
Where: in TraceML's cache folder, traceml/.cache/exports/ inside the opened workspace folder (on the server when you use Remote-SSH, next to traceml/runs). Change it with the traceml.cacheDir setting, as a relative or absolute path. If no folder is open, TraceML uses its VS Code storage folder instead.
<workspace>/traceml/
├── runs/ ← your runs (read-only for TraceML)
└── .cache/ ← written by TraceML only
├── ui-state.json ← remembered view (metrics, selection, filters, columns, charts)
├── runs.sqlite ← run database: fast start + SQL queries (safe to delete)
├── hidden-runs.json ← runs deleted from TraceML (folders untouched)
├── exports/
│ ├── latest.csv ← newest export: give this path to an agent
│ └── table-20261001-102030.csv
└── plots/
├── plot-val-auc-20261001-102512.png ← saved charts (300 DPI)
└── plot-bar-best-val-auc-20261001-102530.csv ← saved chart data
Each export is saved as table-YYYYMMDD-HHMMSS.csv (UTC), and the newest is always also latest.csv, so an agent can simply read traceml/.cache/exports/latest.csv. Only the 20 newest timestamped exports (and the 50 newest plots) are kept. Add traceml/.cache/ to your .gitignore.
Format: UTF-8, comma-separated, a header row and one row per run in table order (RFC 4180 quoting):
id,name,status,group,tags,path,started,duration_s,config.optim.lr,run.best.val/auc,summary.test/auc
20260929-230116_vseg_e9df,vseg,completed,sweep-a,cv;sweep,/mnt/proj/traceml/runs/20260929-230116_vseg_e9df,2026-09-29T23:01:16.000Z,3600,0.0003,0.894,0.881
- The first six columns are always
id,name,status,group,tags(joined with;) andpath(the run directory, so an agent can readmetrics.jsonl,config.jsonorrun.jsonfor more detail). - The other columns follow the table. Logged values are named by kind:
config.<key>(params.yaml),run.start.<metric>/run.last.<metric>/run.best.<metric>(the metric history) andsummary.<key>(final values). Fixed columns keep their names (started,duration_s, …). Timestamps are ISO-8601 UTC, and empty cells are missing ornullvalues. - The export is built by the extension from its own run data; the panel only says which runs and columns are shown. Run data is never modified.
Exporting chart data
Every chart has a 🗎 button that saves the data behind it as a CSV file in traceml/.cache/plots/, so you can pass the chart (e.g. a bar plot) to an agent as plain text. A bar then shows the path with Copy path and Open. Columns use the same names as the table export (config.*, run.best.*, summary.*, …):
| Chart | CSV columns |
|---|---|
| Line | run,path,step,epoch,time_s,<metric>: every logged value of the metric for the plotted runs, full history (not downsampled), empty = null. With Plot by groups: group,step,mean <metric>,n_runs |
| Bar, one per run | run,path,[group:<field>],<field>: all charted runs (selected, or filtered when none are selected), also those beyond the 60 bars drawn |
| Bar, group mean | group:<field>,n,mean <field>,min <field>,max <field> |
| Scatter | run,path,<x>,<y>,[<size>],[group:<field>]. With Plot by groups: group,n,mean <x>,std <x>,mean <y>,std <y> |
| Box | group:<field>,run,path,<field>: one row per run; quartiles follow from the values |
run,path,group:config.data.dataset,run.best.val/auc
baseline,/mnt/proj/traceml/runs/20260929-105832_baseline_d710,cifar10,0.895
high-lr,/mnt/proj/traceml/runs/20260929-111332_high-lr_ad4a,cifar10,0.888
Security and resource use
What the extension can and cannot do:
- Read-only for your data. TraceML never creates, modifies, renames or deletes anything in your runs. Delete only hides a run in TraceML; its folder stays as it is. The only files TraceML writes are in its cache folder (
traceml.cacheDir, defaulttraceml/.cache):ui-state.json,runs.sqlite(written to a temp file, then renamed),runs-index.json(only when SQLite is unavailable),hidden-runs.json, the CSV exports inexports/(newest 20 kept) and the PNG plots and plot data CSVs inplots/(newest 50 of each kept). File names are fixed or generated from a strict pattern, and pruning only removes files matching that pattern.- The run database, the run index and the deleted-runs list are validated field by field when they're read back, and they never decide where TraceML reads: run folders always come from listing the configured runs folders, with the usual symlink checks. Saved plots are checked to be PNG images before they're written.
- Outside the cache module, the extension source uses only
stat,readdir,realpath,readFileandopen(…, 'r'). A test suite (test/security.test.ts) fails the build if a write, process or network API is added anywhere else. It also checks that the cache module writes only fixed file names inside the cache folder. - Files opened from the UI (Files in the inspector) are shown through a read-only
traceml-readonly:file system, so they can't be edited or saved through TraceML.
- Confined to your runs folders. TraceML reads only inside the folders listed in
traceml.roots, and it doesn't search the rest of the workspace.- Every read first resolves symlinks. A run directory or file whose real location is outside its runs folder is ignored and reported (
symlink points outside the run folder). - Open file only accepts a known run plus one of six fixed file names, so a request like
../../secretis rejected.
- Every read first resolves symlinks. A run directory or file whose real location is outside its runs folder is ignored and reported (
- Restricted Mode: because it never executes anything from the workspace, TraceML is declared safe for untrusted workspaces.
- No network, no processes. It makes no HTTP requests, contacts no service (W&B included), and never starts a child process or runs code from the run files.
- Locked-down webview. The UI runs in a VS Code webview under a strict Content-Security-Policy: scripts need a nonce, and there are no remote resources, CDNs or inline event handlers.
- Values from run files (names, params, tracebacks, config) are inserted as text or HTML-escaped. They're never executed.
- The webview can't touch the filesystem. It sends a fixed set of typed messages, and the extension validates each one.
- Untrusted run files are handled defensively.
- JSON, YAML and JSONL parsing never executes anything. YAML alias expansion is capped (billion-laughs protection).
run.json,params.yaml,metrics.jsonandconfig.jsonover 16 MB are not parsed.metrics.jsonlis read up totraceml.maxMetricsFileMB.- A corrupt file only produces a warning on that run.
- Filesystem load is bounded:
- Polling only runs while the TraceML panel is open and visible. Nothing is read when it's closed or in a background tab; it refreshes as soon as you switch back. The default is every 5 s. Finished runs are only re-checked on Refresh.
- Each poll reads each configured runs folder once, plus the core files of the running runs in it. Unchanged files are skipped by modification time and size. On startup the run index avoids re-parsing unchanged runs.
- On slow network filesystems (e.g. NFS on a cluster), raise
traceml.refreshIntervalSecondsand narrowtraceml.roots.
- Dependencies: at runtime only
yaml,uplotandsql.js(SQLite compiled to WebAssembly; its.wasmis loaded from the extension's owndist/folder as bytes, never fetched), bundled intodist/.npm auditreports 0 known vulnerabilities. - CPU: the extension host runs JavaScript on a single thread, so parsing never uses more than one CPU core. A thread limit wouldn't change anything. Big files are read in 8 MB chunks, and
traceml.maxMetricsFileMBcaps the total per run. - Supply chain: a
.vsixruns with the same permissions as VS Code itself, like any extension. Share it from a trusted place (e.g. your team's repository or releases). Before publishing to the Marketplace, set a realpublisherandrepositoryinpackage.json. - Dev-only scripts (
scripts/, not included in the.vsix):fake_runs.py --cleandeletes its output directory. It refuses when that directory contains anything that isn't a run.
Limitations
- Series are loaded only for selected runs, at most 30 at a time. A downsampled series is re-sent in full when it grows (the host sends at most
maxPointsPerSeriespoints per series). - If
metrics.jsonlis rewritten in place with exactly the same size, TraceML doesn't notice until it grows or shrinks. Loggers only append, so this doesn't happen in practice. - A final line with no trailing newline is treated as partial. It shows up once the newline is written.
- The logs viewer, artifact/checkpoint browser, git diff viewer, experiment grouping, filesystem watcher and adaptive polling are not part of V1.
- Only the configured runs folders are read. Runs nested deeper than one level, or symlinked from outside a runs folder, aren't shown.
Fake run generator
scripts/fake_runs.py needs only the Python standard library: no PyTorch, CUDA, W&B, DVC or network access. It writes the exact TraceML format using atomic dot-file-and-rename writes, just like a real logger.
python scripts/fake_runs.py # 24 runs into test-data/traceml/runs
python scripts/fake_runs.py --runs 100 --output test-data/traceml/runs
python scripts/fake_runs.py --runs 1000 --clean # table performance
python scripts/fake_runs.py --big-run-lines 200000 # adds a run with a 200k-line metrics.jsonl
python scripts/fake_runs.py --runs 0 --live # only a live run, until Ctrl+C
python scripts/fake_runs.py --live --live-interval 1 --live-epochs 50 # live run that completes
| Option | Default | Meaning |
|---|---|---|
--output DIR |
test-data/traceml/runs |
The runs directory to write into. |
--runs N |
24 |
How many historical runs to generate (0 means none). |
--seed N |
0 |
Random seed. |
--clean |
off | Deletes the output directory first. It refuses if the directory contains anything that isn't a run. |
--big-run-lines N |
0 |
Also writes one run with about N metrics.jsonl lines. |
--live |
off | After generating, starts a live run that keeps appending metrics. |
--live-interval S |
2 |
Seconds between live epochs. |
--live-epochs N |
0 |
Stops after N epochs and marks the run completed. 0 means run until terminated. When the live run is stopped with Ctrl+C, it's left as running, so it turns stale. |
--no-partial-writes |
off | By default the live run sometimes writes half a line, pauses, then finishes it, to exercise partial-line handling. This option turns that off. |
The historical runs always start with a fixed set of special cases, followed by random parameter variants:
baseline,high-lrandsmall-model(completed)failed-run(CUDA OOM) andnan-loss(failed)killed-run(killed)stale-run(running, but the heartbeat is old)legacy-run(norun.json, nested params)future-schema(schema_version: 2)malformed-lines(a broken line and a partial final line)
For the analysis views, the runs also vary in:
data.dataset(cifar10,cifar100,tiny-imagenet), which clearly shifts the metric distributions. Try Group bydata.datasetwith the box view.model.patch_size, which only exists forvit-tinyruns, so some runs have no value for it.- the
test/*summary values, which are occasionallynull. - the non-numeric summary field
best_checkpoint.
The live run appends to metrics.jsonl, rewrites metrics.json and updates heartbeat_at in run.json for every epoch.
Development
You need Node.js 20 LTS (or newer) and npm on your local machine. Nothing is built on the training server.
node --version && npm --version
npm install
python scripts/fake_runs.py --clean # local fixture data in test-data/traceml/runs
npm run build # esbuild -> dist/extension.js, dist/webview.{js,css}
npm test # vitest
npm run typecheck # tsc --noEmit (strict)
npm run check # typecheck + test + build
npm run package # -> traceml-<version>.vsix
- Debugging: open this folder in VS Code and press F5 (Run TraceML (fake runs)). This opens an Extension Development Host on
test-data/. Then run TraceML: Show Experiments. Startpython scripts/fake_runs.py --runs 0 --livein a terminal to watch a live run. - Previewing the webview without VS Code:
npm run build && npm run preview -- test-datawritestest-data/preview.html. Open it in a browser; it uses the real host-side parsing and a mocked VS Code API. Add#demoto the URL to pre-select a few runs. - Benchmarking the host pipeline:
npm run bench -- test-data/perf(afterpython scripts/fake_runs.py --runs 1000 --output test-data/perf/runs --big-run-lines 200000). On a typical Windows laptop, parsing 1,000 runs takes about 0.7 s, a no-change poll takes about 20 ms, loading a 200k-line history takes about 0.4 s, and re-polling it when nothing changed takes under 1 ms.
Project layout:
src/extension.ts activation, commands, webview panel, polling loop, message handling
src/discovery.ts resolves the configured runs folders and lists the runs inside them
src/fsGuard.ts read confinement (realpath + inside-folder checks)
src/readonlyFs.ts read-only file system used to open run files
src/runStore.ts parsed-run cache (mtime/size), summaries, on-demand histories and details
src/runIndex.ts run state (de)serialization + field-by-field validation (runs-index.json fallback)
src/runDb.ts run database (runs.sqlite via sql.js): schema, incremental updates, load
src/cacheStore.ts the only writer: ui-state, run index, deleted runs, CSV exports, PNG plots
src/model/tableExport.ts, plotData.ts, png.ts, hiddenRuns.ts table/chart CSV, PNG DPI stamping, deleted runs
src/poller.ts non-overlapping interval poller
src/protocol.ts typed host <-> webview messages + validation of webview input
src/types.ts, paths.ts shared types; OS-independent path/glob helpers
src/parsers/* run.json, params.yaml, config.json, metrics.json, incremental metrics.jsonl
src/model/* status/summary, columns, sorting, filters, comparison, downsampling, EMA
src/model/analysis.ts grouping, scatter/box data extraction, quartiles, axis ticks (pure, tested)
src/webview/* vanilla-TS UI: shell.ts (sidebar/header/inspector host), ui.ts (icons, buttons,
chips, switches, popovers, empty states), filters.ts (runs toolbar), table.ts
(virtualized, grouped, keyed rows), plots.ts (metric toolbar + uPlot lines),
analysis.ts (Line|Scatter|Box, SVG scatter/box), detail.ts (inspector),
pngExport.ts (300 DPI chart rendering),
comparison.ts, overview.ts, styles/ (design tokens + CSS)
test/* vitest suites + fixtures
scripts/fake_runs.py fake run generator; scripts/traceml_logger.py reference logger;
scripts/preview.ts, bench.ts are dev-only helpers
Architecture decisions
- No server. There's no HTTP server, no localhost port and no background process. The UI is a VS Code webview.
- No database server. Parsed state is an in-memory cache rebuilt from the files.
traceml/.cache/runs.sqliteis an SQLite file copy of that cache (with each file's mtime and size), used to start faster and for SQL queries. The run folders stay the source of truth; the file can be deleted at any time. - Filesystem only. The run directories are the single source of truth.
- The webview is embedded in VS Code and is a pure view. It can't touch the filesystem. It asks the host for data through a typed message protocol (
src/protocol.ts), and the host validates every incoming message. File opening only accepts a known run plus a fixed list of file names, so the webview never has arbitrary filesystem access. - The extension host owns all filesystem access and parsing. With
extensionKind: ["workspace"], that happens on the Remote-SSH host. metrics.jsonlis the canonical history. It's read incrementally by byte offset, and only for runs being plotted.metrics.jsonis a derived fast summary. The table is built fromrun.json,params.yamlandmetrics.jsonwithout readingmetrics.jsonl. TraceML also reads the first 64 KB ofmetrics.jsonl(once the file is past 64 KB, never again) for each metric's starting value and, for runs withoutmetrics.json, the metric names.- No DVC or DVCLive dependency, and no MLflow.
- TraceML never accesses W&B. The training-side logger may send the same run to W&B on its own; TraceML reads only the local files.
staleis derived from the heartbeat. It's never stored.- Downsampling uses min/max-preserving buckets. The first and last points are always kept, and so are nulls inside a bucket, so gaps survive. It never takes every Nth point.
- Development happens locally with Node.js 20 LTS and npm. Runtime dependencies are only
yamlanduplot, bundled by esbuild intodist/. - Dark-only webview UI with a token-based design system. Everything is vanilla TypeScript and CSS: no UI framework, no CDN, and a strict CSP. Styles are split into
tokens.css,base.css,layout.cssandcomponents.cssundersrc/webview/styles/. - Scatter and box plots use run-level data only. They are computed in the webview from the run summaries it already has (
run.json,params.yamlandmetrics.json), so changing a visualization never touches the filesystem or readsmetrics.jsonl. They are drawn as plain SVG because uPlot is built for x-ordered time series; line plots still use uPlot. - Three kinds of logged values: config, run and summary. The file format is unchanged (
params.yaml,metrics.jsonl+metrics.jsonlast/best,metrics.jsonsummary). The UI, the Compare view and the CSV export name them the same way. - PNG export is rendered in the webview. Line charts are redrawn off-screen at 300/96 scale, with fonts and lines scaled too; SVG charts are rasterized as vectors. The host checks the image, writes 300 DPI into its
pHYschunk, and saves it in the cache folder. - Grouping is computed locally from run metadata. Nothing about grouping is stored on disk or on a server.
- Fake runs come first. Development and tests use
scripts/fake_runs.pydata before validating against real remote runs.
Decisions made where the spec was ambiguous:
exitfailure details: both the flat layout (exit.error_type,exit.error_message,exit.traceback) and a nestedexit.errorobject are accepted.- A running run without
heartbeat_atis shown asrunning?. - The run color is assigned when a run is first selected, using the least-used palette entry among the selected runs, and is kept after that.
- The metric picker lists the metrics of the selected runs, or of all runs when nothing is selected. Until you pick metrics yourself, it defaults to
train/loss,val/lossandval/aucwhere they exist. traceml.rootslists folders explicitly. There's no recursive or pattern-based search, so TraceML never reads anything outside those folders.- A plain number in a text column's filter is a substring match. In a numeric column it's an equality match.
- If a file becomes unreadable (corrupt), the last good parse is kept and a warning is shown.
- In grouping, a run's tags are joined into one label (e.g.
cv, sweep), so each run belongs to exactly one group. Missing values form the(none)group. - Group colors come from the order of the groups among the filtered runs, so they can shift when the filters change.
- A scatter point whose Size value is missing is still plotted, at the default size. Only a missing X or Y excludes it.