vs-image-viewer

English | 简体中文

A VS Code image viewer for PNG, JPG/JPEG, RAW/CFA, RGB-IR, and YUV. Files are grouped by extension in the left sidebar, images open in a zoomable editor, and parsing settings appear in the right Secondary Side Bar.

Installation and usage

Requires VS Code 1.106 or later, with support for extension view containers in the Secondary Side Bar.

• In the Extensions view, choose … → Install from VSIX… and select vs-image-viewer-x.x.x.vsix generated by this project.
• For development, open this directory in VS Code and press F5. Select the image viewer launch configuration. No npm dependencies or compilation are required.
• Click the image icon in the Activity Bar, then + to import files. Selecting a file opens its preview and properties.
• Opening .jpg, .jpeg, .png, .raw, .y, .yuv, or .uv files in Explorer uses this viewer by default and adds them to the file list. Explicit user or workspace workbench.editorAssociations settings take precedence.
• The Properties toolbar button toggles the image properties sidebar. The editor title action and Command Palette command provide the same behavior.
• Properties follow the active editor tab. Switching to a text editor or closing all editors clears the panel; returning to an image restores it. Focusing a properties input keeps the current image selected.
• Other files can be opened through the Explorer context menu or Reopen Editor With.

The imported file list is saved per workspace; per-file settings are kept only for the current session. Extensions are normalized to lowercase for grouping; PNG, JPG, and JPEG have separate groups. Removing an entry closes its previews without deleting the source file. PNG/JPEG metadata is read-only; binary image parsing and display settings are editable.

Filename inference on first open

New binary files initialize their properties from explicit case-insensitive tags in the filename. Tags can appear in any order, separated by _, -, spaces or dots. Only the basename is inspected; parent directories are ignored. Restarting VS Code or reloading its window discards previous per-file settings and infers properties again from the filename. Manual adjustments remain in effect when switching or reopening files within the same session. PNG/JPEG properties still come from the encoded image.

For example, capture_BGGR_1920X1080_12BIT_IN16_MSB_BE.RAW selects BGGR, 1920×1080, 12-bit samples in 16-bit containers, high-bit alignment and big endian. capture_NV12_1920x1080.yuv selects the NV12 preset; capture_P010_1920x1080.bin selects 10-bit semiplanar YUV in 16-bit containers with high-bit alignment.

Explicit fields override preset defaults. Missing or conflicting tags retain the default for that field; inferred invalid values are shown by the usual property validation. Without a container tag, packed uses the pixel depth as container size, while ordinary RAW uses the next whole-byte size. RAW12 alone does not imply MIPI RAW12 packing. LSB/MSB describe alignment inside the container; LE/BE describe byte/bit order. A RGBIR label without a four-character tile cannot determine the array. .y and .uv retain their plane-only meaning. No image-content or file-length guessing is performed.

RAW / CFA / RGB-IR

Unknown extensions default to CFA. Use Image Format to switch to YUV. Width and height are editable and are inferred from names containing dimensions such as 320x240.

• Bayer presets include RGGB, BGGR, GRBG, and GBRG. Other valid R/G/B arrangements are supported.
• CFA patterns are fixed at 2×2. Edit the four cells at the far right of the editor toolbar. Each cell accepts one character and converts lowercase to uppercase. Patterns must contain R, G, and B. Invalid characters trigger a 1.6-second warning animation, followed by a dashed border and dotted background until corrected.
• Sensor type is inferred from the pattern. A pattern containing I is RGB-IR and must contain exactly one I, for example RG / BI. Use the arrangement specified by your sensor documentation.
• The toolbar grid appears only for CFA images and synchronizes with Array Pattern in the properties panel.
• Legacy tile dimensions are ignored. Old patterns with more or fewer than four cells must be replaced with a valid 2×2 pattern.
• Display Channel, on the right side of the toolbar, offers RGB, IR, alpha blend, and side-by-side RGB/IR views.
• IR-only extracts one IR sample per complete 2×2 tile, producing half the original width and height. Incomplete tiles at odd-sized edges are discarded. RGB, blend, and each side-by-side view use the original dimensions.
• In blend mode, an IR Opacity slider appears at the end of the left toolbar group: 0% shows RGB only, and 100% shows IR only.
• Black level, white level, exposure, and gamma are adjustable. A white level of 0 uses the maximum value for the pixel depth.

Image reconstruction design

JPEG preview edges and corners

JPEG previews enable JPEG Edge Repair by default. The corner icon in the left toolbar toggles between the repaired preview and the original browser decode. The pixel readout says Preview RGB8 when correction is applied. PNG, CFA and YUV keep their existing rendering paths; source files are never modified.

media/jpeg-preview.js uses luminance gradients and eight rays at 45° intervals to find compatible color regions, including the interior of a corner. Each anchor must also pass a two-dimensional flatness check. A correction requires the decoded chroma to lie between two supported region colors while luminance identifies the pixel's side of the boundary. Only chroma is replaced; decoded luminance is preserved within RGB8 rounding. Conflicting hypotheses, clipping, low-contrast edges and insufficient evidence retain the original pixels. This is a conservative preview estimate, not recovery of information discarded by JPEG; it does not remove all compression artifacts or sharpen arbitrary textures. Its safeguards are independent of the CFA demosaic coefficient file.

JPEG regression tests compare every pixel of independent JPEG decode snapshots with the original color bars and ColorChecker PNGs. They also cover eight corner orientations, clean edges, smooth ramps, fine texture, alpha and cancellation during file switches. Processing yields between row batches, and toggling always starts from the original decode.

Data flow and responsibilities

Binary preview separates storage interpretation, reconstruction and display. media/core.js reads samples using container size, pixel depth, alignment, byte order, offset and pitch. CFA values are normalized by black/white levels before interpolation. Exposure, gamma and RGB8 output conversion follow reconstruction and any RGB/IR blending; changing preview settings never writes back to the input.

The PNG/canvas preview is RGB8, while binary pixel inspection reports the original 4–32-bit integer sample. Color reconstruction errors are tested on decoded pixels before viewer zoom or browser scaling. Ordinary PNG/JPEG files use their encoded image rather than CFA reconstruction.

Bayer reconstruction

1. Reconstruct green. The four standard Bayer patterns use Hamilton–Adams horizontal/vertical gradients and second differences. Missing red/blue components use axis or diagonal estimates according to their CFA phase.
2. Refine color differences. Green and the opposite chroma channel guide red/blue weights. This detects color boundaries even when green is constant. Edge confidence compares the discontinuity with variation on either side; smooth or uncertain areas retain the initial estimate.
3. Preserve measured data. Interpolation leaves each measured channel unchanged before display transforms. Phase-preserving reflection handles borders, a sample-step floor stabilizes weights, and frozen guides avoid scan-order feedback. Chroma guides use a three-row cache.
4. Correct supported corners. The flat-region corner model below prevents two intersecting edges from being averaged into a blurred corner. Other custom CFA layouts and single-row/single-column images retain general neighborhood interpolation.

References: IPOL's analysis of directional and residual demosaicing and the intra/inter-channel edge-preservation principles in Boccuto et al. (2024). The implementation is a lightweight adaptation, not a reproduction of the paper's iterative regularization solver.

RGB-IR reconstruction order

A tile such as RG / BI cannot become standard Bayer merely by changing the I label to G. Its visible preview follows this order:

1. Reconstruct green from visible RGB samples, including at I positions; resample onto a standard Bayer lattice anchored at the original R phase.
2. Apply the Bayer directional reconstruction to that virtual mosaic.
3. Compute differences between reconstructed and measured visible samples, then interpolate these residuals on the original channel lattices. Bound the corrected estimates by neighboring original same-channel samples to suppress overshoot and alternating row/column stripes.
4. At a sharp, straight edge with flat, stable sides, use the original 2×2 sample cell to assign ambiguous estimates instead of creating an intermediate-color stripe. Smooth gradients and unstable neighborhoods retain existing interpolation.
5. Validate corners against original RGB samples, then apply their correction after resampling, residual transport and straight-edge correction. Otherwise those stages can blur the corner again. Restore measured RGB values at their original sites.

IR values never substitute for green and do not participate in visible edge or corner confidence. IR-only extracts the original half-resolution plane; blending and side-by-side views retain their separate IR interpolation. All visible modes share the same reconstructed RGB.

Corner model and reconstruction limits

Corner detection supports boundary rays at 0°, 45°, 90°, 135°, 180°, 225°, 270° and 315°, including 45°, 90° and 135° corner openings. Axis-aligned corners first use four 4×4 quadrants. Oblique candidates use a 12×12 window partitioned by a pair of boundary lines, with both assignments of samples lying exactly on a diagonal tested against the data. Each sector needs at least two samples of every visible channel. Three flat regions must agree on a background color, and the fourth must differ in at least two channels. A third distinct level in a visible channel rejects the oblique two-color model early. The decision uses raw samples, with no fixture names, known patch coordinates or reference colors in the decoder.

For axis-aligned corners, Bayer correction covers a 6×6 neighborhood and RGB-IR covers the validated 8×8 neighborhood. Oblique correction covers 8×8 pixels within its 12×12 evidence window, accounting for the larger footprint of rotated interpolation. The precheck compares the center with its same-phase neighbors so an acute tip is detected even when both endpoints lie in the background. RGB-IR searches additional visible phases around an I anchor and excludes I from all model checks. Conflicting models at equal priority leave the existing estimate unchanged. Correction strength follows the direction/strength policy; uniform = 1 disables this sharpening.

An I position has no direct RGB sample. Equally supported RGB-IR corner positions prefer boundaries aligned with the original 2×2 cells, consistent with the straight-edge convention. This is an explicit reconstruction assumption: the true edge can remain ambiguous by one pixel. Flat-region fitting does not imply exact recovery of arbitrary textures or diagonal edges. No sensor-specific IR crosstalk correction, white balance, learned model or color calibration is applied.

Conservative anti-alias refinement and display reduction

RAW previews appear in two stages: fast local interpolation first, followed by the full-quality reconstruction. The replacement preserves zoom and pan; playback advances only after the final frame is ready.

Green reconstruction now checks horizontal/vertical color-difference consistency over a 5×5 neighborhood before blending disagreeing candidates. The refinement fades with directional confidence and preserves low-disagreement areas, exact plateaus, evidence-backed corner neighborhoods and strong opposing red/blue transitions. It retains the eight-ray direction policies and every measured sample. Temporary filter planes use 256×256 tiles with an eight-pixel halo; translation tests cover tile seams. This is a conservative GBTF-inspired refinement, not a full ARI implementation.

For RGB-IR, the virtual Bayer image carries a measured/reconstructed confidence mask. The new stage does not directly revise green at synthetic red/blue centers, and synthetic neighbors contribute less to its direction scores. IR values remain excluded. The optional antiAlias object in the coefficient file controls enabled, greenThreshold, chromaThreshold, directionFloor, directionSpan, and syntheticWeight; hover descriptions explain the defaults. Older files without this object inherit the defaults. Set antiAlias.enabled to false to compare with the previous reconstruction.

Fit reduction integrates the covered source-pixel area into a separate display canvas, accounting for device pixel density and premultiplied alpha. Native pixels and inspection values remain unchanged. Integer zoom uses pixel-exact rendering; fractional enlargement uses smooth rendering. The full-frame benchmark, limitations and reproduction commands are in the quality report.

Direction and strength coefficients

The toolbar's sliders icon opens demosaic-coefficients.json beside the image. This file lives in the extension's VS Code global storage directory, survives extension updates, and applies to all images in that VS Code profile. Save to re-render open CFA images. Invalid JSON or coefficients keep the last valid configuration and report an error; the editor provides completion, ranges, and field descriptions. Delete the file to restore defaults; opening it again recreates it from media/demosaic-defaults.json.

Eight independent ray groups cover 0°, 45°, 90°, 135°, 180°, 225°, 270°, 315° (clockwise in image coordinates: right, down-right, down, …). Detection compares like CFA phases two pixels apart and normalizes diagonal distances. Red/blue refinement also uses frozen green and opposite-chroma guides, keeping vector differences separate to avoid cancellation. Tied directions share their strategies. A gradient points across an edge; tangent preference favors samples along it. CFA sampling still constrains available interpolation candidates: horizontal/vertical green candidates, axis or diagonal red/blue candidates.

Each directions.<angle>.levels array contains 2–16 increasing gradient thresholds starting at 0. The supplied weak/middle/strong thresholds are 0, 0.004, 0.016; strength is measured after black/white normalization, before exposure and gamma. Coefficients interpolate continuously between thresholds and use the last level beyond the highest threshold. Defaults are rotationally symmetric; groups can be tuned independently.

These coefficients affect the four standard Bayer patterns and the RGB-IR visible reconstruction path. Measured samples remain unchanged. They tune a lightweight edge-aware algorithm, not a learned or sensor-calibrated reconstruction.

Full-frame regression criteria

ColorChecker tests decode all six Bayer RGGB / RGB-IR RGBI variants: 12-bit in 16-bit containers, packed 12-bit and 16-bit. Each result is compared with the same PNG reference at the original 640×432 dimensions and default display/coefficient settings.

• Check dimensions, buffer length and opaque alpha, then all 276,480 pixels / 829,440 RGB channel values, including every corner, gutter and outer border. No cropping, subsampling or excluded edge regions.
• Require maximum absolute channel error ≤ 1 and mean absolute error ≤ 0.1, both in RGB8 levels. A low average cannot hide a damaged corner. Failure reports the worst pixel coordinate, channel, actual value and expected value.
• Current fixtures have zero error across the full frame. This result describes these synthetic RGB8-derived fixtures, not accuracy on all sensor captures or native 12/16-bit color precision.

Bayer tests also cover phase, boundary parity, contrast reversal, diagonals, gradients and textures. RGB-IR tests cover all 24 tile permutations, IR-signal independence, whole-rectangle reconstruction, ambiguous corners, coefficient behavior and consistency across visible display modes. Original sample preservation remains a separate requirement. Directional corner tests cover all eight orientations, three opening angles, four Bayer phases and all 24 RGB-IR arrangements, plus high bit depths, reversed contrast, IR independence, coefficient control and transposition. Their 48×48 polygon images are checked in full (RGB8 MAE ≤ 1.5, maximum channel error ≤ 100), with a stricter ≤ 1 limit in the central 8×8 corner region. Distant diagonal edges retain the existing interpolation; these limits are separate from the unchanged ≤ 1 full-frame ColorChecker requirement.

Run the reconstruction regressions with:

node --test test/demosaic.test.js test/rgbir-demosaic.test.js test/directional-corners.test.js test/colorchecker-fixtures.test.js

YUV

Presets include I420, YV12, NV12, NV21, I422, NV16, NV61, YUYV, UYVY, I444, NV24, NV42, P010, P012, and P016. Supported matrices are BT.601, BT.709, and BT.2020, with full or limited range. Chroma uses nearest-neighbor upsampling.

Hover over Color Matrix or Range for one second to read the documentation. Keyboard focus also opens it; Esc closes it. The help explains matrix selection, 8/10-bit ranges, and the effects of incorrect settings. BT.2020 uses the non-constant-luminance matrix only, without HDR transfer-function decoding or display gamut conversion.

Display Channel offers color, luma-only, and chroma-only views. Luma ignores U/V; chroma displays U/V colors at a fixed 50% luma. Display selection does not change frame sizes or boundaries. Chroma is unavailable for 400 images.

File Contents selects the actual planes present: full YUV, Y only, or UV only. .y files default to standalone 400 luma; .uv files default to standalone 420 interleaved UV. UV files support 420/422/444 sampling, UV/VU interleaving, and separate U/V or V/U planes without requiring Y data.

Always enter the full image dimensions: a 320×240 image with 420 UV data contains 160×120 U/V sample pairs. Standalone planes support pixel depth, container size, offsets, pitches, and sequences; frame sizes count only the planes present.

For odd dimensions, chroma dimensions round up. The last packed 422 pixel pair still occupies four samples. For example, an 8-bit 3×3 420 frame contains 9 bytes of Y and 4 bytes each of U and V, totaling 17 bytes.

Pixel depth and storage

CFA and YUV samples support unsigned 4–32-bit integers.

• Pixel Depth specifies the number of meaningful bits per sample.
• Container Size specifies the stored bits per sample, from 4 to 32, and must be at least the pixel depth. It determines the storage layout without a separate storage-mode selector. Auto normally chooses the smallest whole-byte container that fits; legacy bit-packed settings remain compatible.
• Bit Alignment appears directly below Container Size only when the container is larger than the pixel depth. Zero-fill high bits places the value in the low bits. Zero-fill low bits shifts it into the high bits. An 8-bit pixel in a 10-bit container can therefore be stored directly or shifted left by 2 bits.
• Byte Order supports Little Endian and Big Endian. Non-byte containers form a continuous bitstream within each row; rows start on byte boundaries and include end-of-row padding. P010 uses little-endian 16-bit containers with 10 meaningful high bits.
• Row pitches default to automatic calculation. The decoder retains support for explicit pitches including padding. The Y/CFA/packed pitch input is hidden; chroma-plane pitch remains editable. Interleaved UV pitch covers the entire UV row. Padding on the last row counts toward frame size.

Pixel Depth, Container Size, and Bit Alignment have hover documentation. Hover over a label for about one second, or focus it using the keyboard; press Esc to close the help.

Bit packing is a per-row contiguous stream, not MIPI CSI-2 RAW10/12/14 grouped packing. v210, signed or floating-point samples, compressed RAW, and vendor container formats require conversion before opening.

Reading and interaction

The viewer reads one frame starting at Read Offset, in bytes. Insufficient data produces inline errors instead of drawing an incomplete frame. Single-frame mode ignores and reports trailing bytes. Read Offset has hover and keyboard-accessible documentation.

Image sequences

Playback controls float over the image with a translucent glass background. They fade out after 3 seconds without pointer or keyboard activity, while playback continues. Moving within an image preview or the properties panel restores them; returning focus to VS Code also restores them. Hovering anywhere over the playback bar, dragging the timeline, or editing frame/FPS values keeps the controls visible. The three-second countdown resumes after the pointer leaves and editing ends. The extension cannot observe every mouse movement in native VS Code areas such as Explorer through the public API.

RAW/CFA, RGB-IR, and YUV files can contain consecutive frames with identical dimensions and layouts. Playback controls appear when the available data after the offset is strictly greater than two frame sizes. Exactly two frames remain in single-frame mode; two frames plus a partial tail allow browsing the two complete frames.

• Frame size depends on container size, sampling, layout, and pitches. Frame N starts at offset + (N - 1) × frameBytes.
• Frame numbers start at 1. Seek with the timeline or enter a frame number, confirming with Enter or by leaving the field.
• Previous/next buttons move one frame and pause playback. They are disabled at the corresponding boundary. Rapid clicks accumulate the target position.
• Playback supports 0.1–120 FPS, defaulting to 24, retained per file for the current session. Space toggles playback when the canvas is focused.
• Seeking pauses playback. Playback stops at the last frame; Replay restarts at frame 1. Hiding a preview pauses it.
• Changing parsing or display settings pauses playback and returns to frame 1. Refresh checks file size again and clamps the frame number to the valid range.
• Frames have no additional headers or gaps. Incomplete trailing data is excluded from the frame count.

Playback reuses a decoder worker, requests one frame at a time, and retains the latest frame's raw bytes for pixel inspection. It does not cache entire sequences or queue a backlog of decoding work. FPS is a target maximum; slow decoding reduces playback speed without skipping frames.

Local binary files use seek-based range reads and worker decoding. Virtual files use the VS Code filesystem API. The extension imposes no fixed image-dimension, pixel-count or file-size limit. Actual capacity depends on available memory and the runtime/browser. Side-by-side RGB/IR doubles the output width. Local binary files read only the selected frame; virtual files require a full-file read before selecting the frame. Invalid or numerically imprecise dimensions and ranges are still rejected, and insufficient file data is reported inline.

Preview controls

The glass toolbar uses compact line icons. Keep the pointer still over an icon for two seconds to see its label. Moving, clicking, or dragging dismisses the tooltip; keyboard focus also provides help, and Esc closes it.

• Scroll to zoom around the pointer; drag to pan.
• At 6400% (64×), a pixel grid appears automatically and tracks panning. It hides when zooming out.
• While the grid is visible, translucent top and left rulers show zero-based coordinates. Side-by-side views use source-image columns for each half.
• Fit to Window scales images up or down proportionally to the available viewer area, with a 16-pixel margin and the existing 64× zoom limit. It updates automatically when the viewer resizes. Click the button again to return to 1:1 actual pixels; another click fits the image to the window again.
• With the canvas focused: F fits the image, 1 selects actual pixels, and + / - zoom.
• Pointer inspection shows coordinates and original sample values: Y/U/V for YUV, or the R/G/B/I sample at the CFA location. Values retain their original 4–32-bit precision and are unaffected by exposure, gamma, or demosaicing. Missing planes display —. PNG/JPEG inspection shows RGB8.
• In IR-only mode, coordinates refer to the reduced image and readings come from the corresponding original IR samples. In side-by-side mode, coordinates map to the source image; CFA readings report the stored sample at that location, including in the interpolated IR view. Stale pixel-query results are discarded.
• After changing a source file, refresh it from the preview toolbar or file list.
• All settings affect the preview only. Source files are never modified.

Development and verification

Localization

The extension supports English and Simplified Chinese and follows the VS Code display language. Unsupported languages fall back to English. Changing the display language and restarting VS Code updates commands, properties, hover help, toolbar labels, and errors.

Strings use stable resource keys, similar to Android string resources:

• locales/en.json: English and the fallback catalog.
• locales/zh-cn.json: Simplified Chinese.

For example, code calls t('field.endian'). Dynamic values use named placeholders, such as t('error.integer', { min: 4, max: 32 }); avoid concatenating translatable sentences. Protocol identifiers, channel letters, and units remain unchanged.

To add a language, copy locales/en.json to a language-tagged file such as locales/ja.json, translate its values while retaining keys and placeholders, and run npm run localize. Set demosaic.schema.url to ./locales/generated/demosaic-schema.<language-tag>.json for the new language. Root package.nls*.json files and locales/generated/demosaic-schema.*.json are generated; do not edit them directly. media/demosaic-schema.json keeps validation rules and %demosaic.schema.*% resource keys only. Schema titles and hover descriptions come from the catalogs, and the localized manifest selects the schema for the VS Code interface language. Packaging also regenerates these files. Checks detect stale output, and tests compare resource keys and placeholders.

Preview either language at http://127.0.0.1:4317/?lang=en or http://127.0.0.1:4317/?lang=zh-cn. Manifest localization uses %key% and package.nls.json; runtime localization selects the same catalogs through vscode.env.language.

Commands

Requires Node.js 22 or later. There are no external runtime dependencies.

npm test
npm run check
npm run fixtures
npm run package

Commands can also be invoked directly, for example node --test test/*.test.js or node scripts/check.js.

npm run preview serves a standalone UI test page at http://127.0.0.1:4317, using the actual viewer and properties code with project fixtures. Native VS Code file-tree and extension-host behavior still require F5 verification.

Extension-host integration tests live in test/extension-host.js and can run through VS Code's --extensionDevelopmentPath and --extensionTestsPath arguments. They verify command registration, default editor associations, webview decoding, and sequence recognition.

Fixtures

Added complex scenes: traffic intersection, wildlife, and football, each at 1536 × 1024, with 12-bit-in-16, packed 12-bit and 16-bit CFA variants (nine RAWs) and three RGB reference PNGs. Explicit filenames auto-select the decoding settings. See fixture catalog and reproduction instructions; regenerate with npm run fixtures:scenes. These are synthetic CFA samples derived from AI-generated RGB8 references, not native high-dynamic-range sensor captures.

Added a 640 × 432 ColorChecker 24 IQ chart with a PNG reference and six Bayer RGGB / RGB-IR RGBI RAW variants (12-in-16, packed 12-bit and 16-bit). It contains a 6 × 4 patch grid including six gray levels, using the published 2005 sRGB reference palette. Run npm run fixtures:colorchecker; see the chart catalog, source and test conventions. All variants are included in the standalone preview.

Tests cover pixel depths, container sizes, byte order, alignment, YUV layouts, odd dimensions, RGB-IR views, validation, offsets, worker decoding, PNG encoding, sequence boundaries, seeking, playback timing, FPS changes, and toolbar configuration synchronization.

Disk changes are tracked for imported images, including files outside the workspace. Selecting an existing image or returning to the VS Code window checks its displayed source data and offers Reload / Keep Current Image only when it differs. RAW/YUV compares the displayed frame range, so changes to other frames or appended data do not reload the current image. Timestamp-only changes do not prompt. Unavailable files preserve the current image; accepting Reload rechecks the data in case it was restored while the prompt was open.