WeightScope

A local VS Code extension for inspecting model checkpoints and answering what changed, and are the weights healthy? Original checkpoints stay unchanged.

Open .safetensors (single-file or sharded), .gguf, .onnx, .pt, or .pth files, sharded PyTorch .bin checkpoints, or select a downloaded Ollama model from its local catalog. Explore a layer tree, shapes, stored parameter counts, dtype distribution, quantization metadata, tensor statistics, and checkpoint differences. Inspection and analysis run on demand. There are no uploads, network requests, or telemetry in the extension.

What instruction tuning changed in SmolLM2-135M: every tensor of all 30 layers on one log scale. Layer 0 moved most (its input norm and value projection by about 32%, its output projection by 21%); value and output drift then fades with depth, while query and key drift roughly doubles, from about 6% to 12%.

The model map of a real Q4_K_M GGUF (Gemma 3 4B via Ollama translategemma:4b), read from metadata alone: most projections are Q4_K, some attn_v and ffn_down layers stay in Q6_K, and the vision tower's attn_v mixes Q8_0 and Q5_0.

Install

1. In VS Code, open the Extensions view, search for WeightScope, and click Install. You can also run code --install-extension MariaRajabzadeh.weightscope.
2. Click the WeightScope icon in the Activity Bar. The Start view has buttons to inspect a model file, open an Ollama model, or compare checkpoints, so you don't need a model open first. You can also open a supported model file from the Explorer, or run any WeightScope command from the Command Palette.

If another extension owns these file types, use Reopen Editor With… → WeightScope. Requires VS Code 1.90 or newer. Works locally and with Remote SSH when the extension and model are on the remote host.

A simple first workflow

1. Open a checkpoint. Explore opens on the Model map: every tensor is one cell, with layer groups as columns and tensor roles as rows. Color it by Precision to see a quantization recipe at a glance (for example, which layers a Q4_K_M file keeps in Q6_K), or by Stored values. Click Scan weights to sample every tensor once, then color the map by Weight RMS or Largest |value| to see how weights change with depth. Select a cell to open its tensor. Below the map, aligned layer rows switch between Stored values and Storage bytes; bars share a scale across all pages. Unknown storage sizes remain explicitly unknown.
2. Select a group, then a tensor. Its details open in a focused drawer.
3. Click Draw weight image to see the whole tensor as pixels, or Analyze weights to see a labeled distribution and health check. Distribution bin counts exposes the values without hovering; advanced statistics stay under More statistics & sampling details.
4. Use Compare checkpoint to choose a target, then Analyze weight changes. The Layer drift map aligns layer groups against actual tensor-name suffixes. Its Fingerprint view draws the whole comparison as one dense image: layer groups run left to right and tensor roles top to bottom, so depth patterns show at a glance. Hover, or focus the map and use the arrow keys, to read a cell; click or press Enter for its tensor difference. The Table view shows the same cells with their values written in. Select a row label there for all tensors in that group. Search the map and page through groups and roles to inspect large models.

The map colors finite relative L2 change (||target − baseline|| / ||baseline||) on one scale shared across every page and filter, using a perceptually uniform ramp (reversed in light themes so more drift always has more contrast). The Log scale (default) spans the smallest to the largest positive drift, up to six decades, so a 0.1% change stays visible next to a 50% outlier; Linear scale runs from 0 to the largest drift. Color measures drift, not health. Exact zero drift has its own neutral color. Cells distinguish uncomputed and unavailable analysis, precision changes, non-finite mismatches, and structural changes; empty cells mean no tensor in that group/role. Full and sampled coverage is listed under the map and in each cell's details. A zero baseline norm shows n/a, not zero drift. If normalized names collide, the cell takes the largest drift among them and opens a list of every matching tensor.

The weight image reduces a tensor to at most 192 × 192 tiles and colors each tile by its RMS (default) or its signed peak, the value with the largest magnitude, so a single outlier stays visible. Outlier channels show as bright lines, attention heads as bands, and untrained vocabulary rows as dark regions. Leading dimensions form rows and trailing dimensions form columns, in row-major order (a convolution kernel [out, in, kh, kw] draws as out × in·kh·kw). Tensors up to 4,194,304 values (262,144 for PyTorch) are read in full; larger ones read evenly spaced runs of whole rows, every column, and say how many rows were read. Colors clip the outer 0.5% of tiles by default, or use the full range. Hover a tile for its rows, columns, peak and RMS. In a comparison, Draw change image shows target minus baseline at the same positions, including translated GGUF tensors.

Top: the first query projection of Gemma 3 4B as an RMS image; horizontal bands follow its attention heads. Bottom: the same model after Scan weights, colored by RMS; the post-attention and post-FFN norm weights grow steadily with depth.

The tensor difference drawer overlays baseline bars and a target line with identical bin boundaries and a common vertical scale (percentage of finite analyzed values). It uses the existing bounded samples; no extra weight reads or background scans are needed. NaN/Inf diagnostics remain separate. JSON reports include the histogram bounds and both series.

Every section has a ? button that explains it in place: what it shows, how to read it, and how its numbers are computed and where they stop being reliable. The tensor inspector also names each tensor's likely role from common Hugging Face and llama.cpp naming conventions (for example, an attention query projection) and describes its storage type (for example, how Q4_K packs 4.5 bits per weight). The Guide tab collects every explanation in one searchable page, with research recipes for common questions.

Browse tree opens the tensor name hierarchy. More columns reveals exact values and storage bytes. Tools contains refresh, JSON export, and Python setup. Model details holds full metadata and precision distribution.

Open sharded Hugging Face checkpoints

Large Hugging Face models ship as model-00001-of-0000N.safetensors shards plus a model.safetensors.index.json index. Open the index or any shard: WeightScope reads every shard's header and presents one checkpoint, with combined totals, layer groups, statistics, and comparisons. Sharded checkpoints compare with each other and with single-file, GGUF, ONNX, PyTorch, and Ollama checkpoints. Only shard headers are read when opening; numerical analysis reads each tensor from its own shard.

Older Hugging Face models ship PyTorch shards instead: pytorch_model-00001-of-0000N.bin plus pytorch_model.bin.index.json. They open the same way, from the index or any .bin shard, and follow the same validation and fallback rules below. Shard metadata and values are read through the PyTorch helper with restricted weights-only loading, so .bin shards need a trusted workspace and the PyTorch setup. The helper runs once per shard when opening, and each analysis reads only the shard that stores the tensor. A standalone .bin file that is not a shard (such as a single pytorch_model.bin) is not opened. Many repositories ship both layouts in one folder: each shard uses the index of its own format, and the two checkpoints compare tensor by tensor.

The index must sit next to its shards and may only name files of its own format (.safetensors or .bin) in that folder. Hugging Face cache snapshots (models--<owner>--<name>/snapshots/<revision>) work in place, including their symlinked blobs, and are named after the repository. Model details → Safetensors shards (or PyTorch shards) lists each shard with its tensor count and size, and the tensor inspector names the shard that stores each tensor.

Any index name ending in .safetensors.index.json or .bin.index.json is supported, such as diffusion_pytorch_model.safetensors.index.json. If the index is missing but every shard is present, shards are grouped by file name, with a note. An index that is invalid, names a missing or unreadable shard, or disagrees with the shard contents (a listed tensor is absent, stored in a different shard, or duplicated) is reported as an error when you open the index. Opening one of its shards instead shows that shard alone, with a note that totals and comparisons cover only that file. Tensors stored in shards but missing from the weight map are included, with a note. To view an index as JSON, use Reopen Editor With… → Text Editor.

Open downloaded Ollama models

Click Open Ollama Model… in the WeightScope Start view, or run WeightScope: Open Ollama Model from the Command Palette (Cmd+Shift+P on macOS, Ctrl+Shift+P on Windows/Linux), then select a model/tag. The picker reads the local catalog and opens the original weights in place. Ollama does not need to be running, and no rename, copy, download, or model execution is needed.

You can also open a tag file directly under <store>/manifests/<registry>/<namespace>/<model>/<tag>, such as ~/.ollama/models/manifests/registry.ollama.ai/library/tev1/4b. WeightScope resolves this small JSON manifest to its downloaded weight blobs. If the file is already open as JSON, use Reopen Editor With… → WeightScope, or select the model with WeightScope: Open Ollama Model.

Both GGUF model blobs and newer safetensors tensor blobs are supported. Manifest tensor names are retained, including image model namespaces and shared weight aliases. Local manifest-list references resolve to a downloaded variant, preferring GGUF when available. Model names, tags, registry, prompt template, system prompt, generation parameters, licenses, and small model configurations appear under Model details → Ollama source & prompt settings. Additional adapter/projector files are listed separately and excluded from primary weight totals.

The extension uses weightscope.ollamaModelsPath if set, then the OLLAMA_MODELS environment variable visible to VS Code. Otherwise it searches the default user store (~/.ollama/models) and Linux service stores. Choose Ollama models folder… in the picker selects a different store for this session; choose the folder containing both manifests and blobs. Set weightscope.ollamaModelsPath to keep that location across restarts. For Remote SSH, the store must be accessible on the extension host. See Ollama's storage documentation.

Run WeightScope: Compare Ollama Models to select a baseline and target from the catalog. Inside an inspector, Tools → Compare Ollama model or Compare → Choose Ollama model compares the current checkpoint with a downloaded model. Regular file checkpoints and Ollama models can be compared with each other.

GGUF numerical analysis uses the supported decoders listed below. Ollama MXFP8 safetensors weights support numerical analysis and checkpoint comparison: four E4M3FN values are unpacked from each U32 word and multiplied by their companion E8M0 scale, shared across 32 weights. Decoded values use float32 rounding, including signed zeros, NaNs, overflow, and underflow. The tensor inspector shows the logical shape and weight count alongside a separate stored-layout disclosure. Model totals, size bars, and tensor tables continue to describe stored entries, including scale codes; they are not unique trainable parameter counts. Other packed schemes (including FP4), missing scales, and incompatible MXFP8 layouts expose metadata with an explicit numerical-decoding explanation. Missing/corrupt downloads, unsupported blobs, and cloud-only entries show explanations. Text/configuration previews are limited to 64 KiB. JSON reports can include local prompts and settings; review them before sharing. Export cannot overwrite the selected manifests or their weight/configuration blobs.

Layer groups follow the first numbered module in tensor names, or the enclosing named module when there is no numbered block. Bars describe stored value counts or storage bytes; they do not infer execution order. Both the optional tree and search retain access to individual tensors.

What you can do

• Browse the tensor name hierarchy and search all layers, with dtype filtering and paged results.
• See tensor shapes, stored elements, storage bytes, precision distribution, and model metadata.
• Analyze a selected tensor: min, max, mean, population standard deviation, RMS, zero fraction, NaN/Inf counts, and histogram.
• Draw a tensor, or its change between two checkpoints, as an image of per-tile RMS or signed peak values.
• Compare a baseline and target checkpoint by tensor name. Detect added/removed tensors, changed shapes, and changed dtypes immediately.
• Compare a GGUF file with the Hugging Face checkpoint it was converted from, layer by layer, to measure quantization error. See GGUF and Hugging Face checkpoints.
• Analyze numerical differences for one tensor or a batch: mean absolute delta, maximum absolute delta, RMSE, relative L2, cosine similarity, changed-value count, and non-finite mismatches. The tensor difference drawer and JSON report contain the full metrics.
• Export a JSON report containing metadata, computed statistics, comparison results, coverage, and tolerance.
• View ONNX operators and input/output names.
• Save a validated checkpoint copy under a new name with Tools → Save checkpoint as….

Save a checkpoint copy

Tools → Save checkpoint as… streams a complete copy to a temporary file, validates its tensor structure, and publishes it under a new name. Existing files and aliases are never overwritten. Cancellation and validation failures remove the temporary file. Memory usage stays bounded while copying large models.

Single-file safetensors, GGUF (including downloaded Ollama GGUF blobs), ONNX, and supported PyTorch checkpoints retain their format. PyTorch validation requires the same trusted workspace and interpreter as inspection. ONNX files with external weights must stay in the original directory to preserve relative references; external weights are shared, not copied. Multi-blob Ollama safetensors and sharded safetensors or PyTorch export are not supported; copy the model folder to duplicate a sharded checkpoint.

Format support

GGUF IQ and newer schemes expose their type, count, block geometry, and byte size. Their numerical decoders are not included in this release. Unknown GGML types and version-dependent Q8_1 block layouts show unknown byte sizes. Big-endian GGUF and version 1 are unsupported.

ONNX quantized integer statistics describe stored numeric codes, not dequantized floating-point weights. Comparisons reject mismatched numerical units. Sparse initializers, segmented tensors, string/complex and newer sub-byte types do not have numerical analysis. Sparse initializers are excluded from totals, with a visible warning. ONNX inspection is a focused reader, not a full ONNX validator or inference engine.

PyTorch inspection does not execute forward passes or accept custom pickled modules/TorchScript archives. Quantized PyTorch tensors are not supported by this release's metadata reader; export them as ONNX or GGUF. Setup: PyTorch instructions.

Accurate interpretation

The tree follows tensor names; it does not recover the forward-pass graph from a state dict. Counts are stored tensor elements, including buffers and aliases, not guaranteed unique trainable parameters.

Opening native-format checkpoints reads bounded metadata and skips weight payloads. Large ONNX messages are scanned without materializing raw weights. GGUF metadata arrays show the first 32 entries while preserving their full count. PyTorch metadata uses FakeTensorMode and weights-only loading; numerical analysis maps modern ZIP checkpoints on CPU. Legacy non-ZIP PyTorch checkpoints up to 512 MiB may require an eager load when analyzing values.

Statistics use up to 32 evenly spaced contiguous windows across flattened row-major indices, bounded by weightscope.sampleSize. Small tensors are analyzed in full. Results explicitly say Full tensor or Sampled estimate; sampled extrema and health checks can miss outliers outside the sample.

Comparisons use identical logical sample positions across formats. Sample matches means no differences were found in analyzed positions; it does not prove whole-tensor equality. Full match means every value was compared and agreed within tolerance. The JSON status for a full match remains equal. Precision changes remain visible even when numerical values match. NaN/Inf pairs are counted separately. Equal NaNs and equal infinities agree for this comparison policy; mismatched non-finite values mark the tensor changed. Relative L2 is undefined for a zero baseline norm, and cosine is undefined for zero norms.

Files changed since inspection must be refreshed before analysis. No background model scans or weight analysis occur: Scan weights runs only when you click it, samples each tensor the same way as Analyze weights, skips tensors already analyzed, and can be cancelled. PyTorch scans start the helper once per tensor, so they are slower. Precision colors include block scales in bits per value (GGUF Q4_K is 4.5 bits); sizes and statistics use a log scale spanning up to six decades.

GGUF and Hugging Face checkpoints

When one checkpoint is GGUF and the other is a Hugging Face safetensors or PyTorch file, tensor names are translated the way llama.cpp's convert_hf_to_gguf.py writes them, for example model.layers.0.self_attn.q_proj.weight ↔ blk.0.attn_q.weight. This measures what quantization changed in each layer, such as Q4_K or Q8_0 error against the original BF16 weights. Translation applies to these GGUF architectures: llama (including Mistral and Mixtral conversions), granite, granitemoe, smollm3, arcee, olmo, olmo2, qwen2, qwen2moe, qwen3, qwen3moe, phi3, gemma, gemma2, and gemma3.

Value changes made during conversion are reversed before comparing:

• Rotary Q/K permutation. For llama, granite, granitemoe, smollm3, arcee, and olmo, the converter reorders query and key rows from Hugging Face's half-split rotary layout to llama.cpp's interleaved layout. WeightScope restores the original order using the GGUF attention.head_count (queries) and attention.head_count_kv (keys). Without usable head counts, those two tensors are marked unavailable with an explanation instead of reporting false drift.
• Gemma norm offset. Gemma 1–3 GGUF files store RMSNorm weights as 1 + w; 1 is subtracted from the GGUF values. Adding 1 passed through float32, so a tiny absolute tolerance treats that rounding as a match.
• Stacked experts. ffn_gate_exps, ffn_up_exps, and ffn_down_exps hold every expert in one tensor; each Hugging Face expert is compared with its slice.

Comparison rows use the Hugging Face tensor name. The tensor difference drawer shows both names and every step that was reversed, and search matches either name. Other architectures, ONNX files, and checkpoints in the same format keep exact-name matching; the comparison explains when a GGUF architecture is not supported. Tensors without a counterpart (for example rope_freqs.weight or rotary_emb.inv_freq) appear as added or removed. Tensors the converter trimmed, such as unused vocabulary rows in Gemma 3 embeddings, appear as shape changes. Fused Hugging Face expert tensors (mlp.experts.gate_up_proj) are not split.

Settings

Use Analyze next batch again to continue through large comparisons. Analysis can be cancelled. Reports contain the paths of inspected local files; review them before sharing.

Develop and verify

The extension has no runtime npm dependencies or build step.

npm install
npm run check
npm test
npm run demo
npm run preview
npm run screenshots
npm run package

npm run package writes artifacts/weightscope-<version>.vsix (Node.js 22 or newer is required for the packaging tools); install it with Extensions: Install from VSIX…. Open this folder in VS Code and press F5 to start the Extension Development Host. Open one of the generated files in examples/, or a shard in examples/tiny-transformer-sharded/. The browser preview at http://127.0.0.1:4173 runs the same frontend against real synthetic safetensors checkpoints. Add ?ollama=1 for the generated Ollama catalog demo, or ?sharded=1 for the same weights as a three-shard checkpoint. File dialogs, the catalog picker, export, and cancellation belong to the VS Code integration.

npm run screenshots regenerates the README images from the same preview with headless Chrome (set CHROME_PATH if Chrome is not in its default location). Point it at real checkpoints: WEIGHTSCOPE_SHOT_MODEL for the Explore images, WEIGHTSCOPE_PREVIEW_BASE and WEIGHTSCOPE_PREVIEW_TARGET for the drift map, and optionally WEIGHTSCOPE_SHOT_TENSOR for the weight image. The images above used Ollama translategemma:4b and the model.safetensors files of HuggingFaceTB/SmolLM2-135M and SmolLM2-135M-Instruct, saved under their model names. Without them, it falls back to the synthetic demo checkpoints.

Framework integration verification is optional and uses an isolated environment:

python3 -m venv .test-runtime
.test-runtime/bin/python -m pip install 'torch>=2.6' onnx safetensors gguf
WEIGHTSCOPE_TEST_PYTHON="$PWD/.test-runtime/bin/python" npm run test:integration

On Windows, use the environment's Scripts/python.exe. The test suite covers corrupt/truncated files, sampling semantics, cross-format drift, GGUF decoding, external-data path confinement, file mutation, and framework-written checkpoints. See product direction.

References

Hugging Face ↔ GGUF name translation and the reversed conversion steps follow llama.cpp's convert_hf_to_gguf.py model conversions and gguf-py tensor name map. Parsers follow the upstream safetensors specification, GGUF specification, GGML quantization layouts, and ONNX protobuf definitions. Ollama MXFP8 decoding follows Ollama's packing and block parameters, companion naming and quantization metadata, and the OCP Microscaling Formats specification. VS Code integration uses its Custom Editor API.

MIT licensed.