Image Caption GalleryA fast, local-first VS Code extension for browsing image/video datasets and editing sidecar captions without leaving the editor. Open a folder as an adjustable gallery, select an image or video, and review or edit its matching PreviewGallery overview
Image and Markdown caption
Caption editing
Features
Two-image comparisonCtrl-click (Windows/Linux) or Command-click (macOS) two images in Gallery. Selection order assigns A to the left and reference B to the right; both A/B markers remain visible for 350 ms after the second selection before comparison opens automatically. macOS Ctrl-click also works without a conflicting context menu. Ordinary clicks open details; videos cannot be selected for comparison.
Comparison never reads captions or changes the Gallery Caption preference. A, B and an independent left reference copy are decoded before switching is enabled; prepared switches need no new preview requests or decoding. All three count toward the 128 MiB decoded-image cache estimate, with unrelated entries evicted first. Explicit active originals or animations can exceed the estimate; their quality and animation are preserved. RAW and transparency use the existing host preview pipeline. Animation timing is not synchronized. Each side offers Retry / Load original on failure. Browser and panel operate independently after opening; browser sessions expire after 30 minutes of inactivity. InstallInstall directly from the VS Code Marketplace:
You can also download the latest When using Remote SSH, install the extension in the remote extension host. If you run the command in a remote terminal, copy the Usage
If the matching Settings and small windowsBoth Gallery and details have a Settings button. The drawer groups Caption visibility, reading format, font size, pane layout/share, image zoom, thumbnail size, preview quality, zoom speed and video/FFmpeg options. Each option includes a label or explanation. The detail toolbar shows the file name and original dimensions, plus preview dimensions when resized, alongside Back, Edit and Settings. Display preferences remain in the drawer. The detail toolbar also has a Caption shortcut ( Changes apply immediately and are remembered for the current Gallery, without editing workspace settings. VS Code default settings… opens the extension's settings to change defaults for new galleries. For SSH, FFmpeg paths refer to the server. Labels follow the VS Code display language (English or Chinese). Auto layout places text below the image below 680 px. You can also force side-by-side or stacked panes. The settings drawer scrolls independently and fills narrow screens. Escape closes Settings and returns focus to its button; it does not leave the current media. Caption edits keep the existing Save / Discard / Continue protection. Opening settings preserves video playback; changes to video compatibility prepare a new video source. Preview resolution and image-only browsingConfigure these values in VS Code Settings, or add them to
For Remote SSH, previews are generated on the remote extension host before being transferred to the client. Small thumbnails and detail previews use separate cache entries. A bounded CPU decoder pool prioritizes queued detail images and only requests thumbnails near the viewport. The disk cache uses a 256 MiB LRU budget; files currently in use remain protected, so active images can temporarily exceed that budget. The detail header shows oriented original and preview dimensions beneath the filename, without opening Settings. Settings also retains the dimensions; hover for additional preview information. Image decoding defaults to multiple reusable processes on the extension host, including the server under Remote SSH. In Settings → Image decoder execution / Parallel image decoders, choose processes or Worker threads and automatic or manual parallelism. Both use multiple CPU cores; processes isolate crashes, while threads avoid IPC copies and can be faster. Settings → Current image preparation separates host queue, source read, decode/resize/encode (including decoder startup when cold), cache writing, resource loading and browser decoding. Switch to paint measures the current navigation separately from any earlier resource preparation. JPEGs use scaled decoding when shrinking; RAW uses adequate embedded previews before full sensor development. Once the current request enters the host queue, Gallery prepares one image on each side, overlapping idle decoder work with transfer and browser decoding. Adopting an in-flight neighbour also starts the next neighbour immediately. Foreground images outrank neighbours, which outrank grid thumbnails. Ready images reuse the decoded element; recently viewed images remain in a maximum seven-entry, estimated 128 MiB decoded-memory LRU, with the current image protected. Oversized originals/animations stay on demand; neighbours above 64 MiB estimated RGBA are not retained speculatively. A cached target thumbnail or a labelled previous frame stays visible during a slow switch; a loading indicator appears only after 150 ms. First visits and navigation faster than preloading can still need preparation. Captions are prefetched only while visible, displayed independently of image loading and revalidated without overwriting edits. Image modification time/size and preview settings invalidate cached previews. Unused in-flight work is cancelled on rapid navigation; hidden panels and leaving details release neighbours/history. Videos remain on demand. Oversized images are encoded as WebP previews at quality 85, retaining transparency and GIF/WebP animation. Detail images within the budget use their original files. If preview generation fails or exceeds its resource limits, select Load original in the detail view. This fallback is explicit to avoid an unexpected full-size transfer. A bundled WASM32 ImageMagick runtime handles processing without requiring image tools on the server. Set Large foldersThe gallery renders only visible rows plus a small buffer. Scrolling recycles cards and releases images outside that window; opening a detail or hiding the panel releases grid resources. Only four thumbnail requests can be loading at once. Search covers the entire discovered dataset with a short debounce, and returning from details preserves the gallery's scroll position. Arrow keys, Home/End, and Tab continue through files outside the rendered window. Grid thumbnails use a separate static WebP capped at 512² total pixels, or a lower Directory discovery sends metadata in batches of up to 512 records, so large folders can be browsed while scanning continues. Closing or refreshing a gallery cancels its previous scan. Cache hits and scrolling no longer sort the entire preview cache on each release. JPEG grid thumbnails use scaled decoding, avoiding full-size pixel buffers before resizing. Camera metadata is removed from thumbnails while ICC color profiles are retained. Small WebP thumbnails (up to 256 KiB) are sent in the existing response, avoiding another Remote SSH resource request. The scheduler prioritizes the visible viewport, keeps 4–16 requests in flight according to decoder capacity, and uses one visibility scheduler. Failed or stalled images release their slots, and hover details separate host queue/processing from message and browser display time. Cached previews remain reusable after reopening or restarting. Large PNGs use a bounded 384 MiB pixel cache; automatic process counts reserve about 768 MiB per decoder for pixel caches, WASM and input buffers. For repeatable measurements, run Camera RAW and zoom sensitivityCamera RAW previews are prepared on the extension host. For TIFF-based camera containers with a correctly oriented embedded JPEG large enough for the requested pixel budget, Gallery reuses that camera-rendered preview and decodes close to the target size. Small previews are never enlarged to replace the requested quality. Missing, unsupported, damaged or aspect-mismatched embedded previews fall back to the bundled LibRaw decoder, using camera white balance and sRGB output. Original-resolution requests always use full RAW decoding. Supported suffixes include RAW images are always converted to a browser-readable preview, including when they are below the pixel limit or
Full-screen image browsingOpen an image and click Full screen in the detail toolbar, or press F. The button keeps its text on narrow screens. In the browser, click View full screen (or press F) to enter native fullscreen: only the image and any necessary black borders remain, without VS Code or browser chrome. This second user gesture is required by the browser. Landscape and portrait images fit completely, preserving their aspect ratio without cropping. ← / → switches images without leaving fullscreen; Home / End jumps to the first/last image. Touch swipes also work. The toolbar, arrows and cursor hide after two idle seconds and reappear on interaction; Esc exits fullscreen and stays in the browser. Double-click toggles fullscreen in the browser; Gallery double-click still resets image zoom. Settings → Images → Full-screen image browsing opens the current image, or the first image in Gallery's current search results. The browser follows that image order and skips videos. It never reads captions; Caption visibility and unsaved edits stay in Gallery. The viewer uses the preview quality chosen when opened, including original-resolution mode and RAW decoding. To use a different preview quality, change it in Gallery and open a new viewer. Adjacent images are generated on the host, transferred and decoded before navigation. The browser keeps up to seven images within a 128 MiB estimated decoded-memory budget, protects the displayed image and checks file versions in the background. Oversized originals/animations load on demand instead of speculative downloads; source quality and animation behavior are preserved. Cold images can still take time to process or transfer. SSH uses the existing private forwarding service; closing Gallery keeps the browser session alive, while disconnecting SSH or closing VS Code stops it. Sessions expire after 30 inactive minutes. Video browsingMP4, WebM, MOV, M4V, AVI, MKV, MPG/MPEG, WMV, 3GP, MTS/M2TS, OGV, VOB and FLV appear alongside images. Scanning reads file records only. A visible video card requests a small JPEG of its first frame, with a play icon and format badge; it never loads a video into the grid. Poster generation needs FFmpeg and is cached. Small JPEGs travel directly in the result message, avoiding a second Remote SSH resource fetch; only Gallery’s visibility scheduler controls loading. Cards show preparation or failure status instead of waiting silently. Hover a ready thumbnail for cache status, byte size, host queue/extraction time and approximate message/display overhead. Refresh retries failures. Videos start paused and use native playback, full-timeline seeking and volume controls. Press Space when the detail view has focus to play/pause; focused player shortcuts remain native. Videos share TXT captions and the Caption toggle with images. Switching files releases the source and cancels unused requests. Hiding the Gallery pauses playback and compatibility requests; opening Settings preserves playback. Full screen in the video toolbar ( VS Code disables native HTML video fullscreen in extension webviews (upstream issue). Settings → Video → Viewing area → Fill panel keeps playback inside Gallery, temporarily hides Caption and restores its previous layout on Escape. It remains distinct from browser fullscreen and does not change VS Code layout settings. The browser reads supported originals with byte ranges, including H.264/AAC MP4/MOV. Scaling or compatibility uses bundled HLS.js 1.7.3 and only requested four-second H.264/AAC MPEG-TS fragments; no CDN or whole-file conversion is required. FFmpeg needs Gallery serializes SourceBuffer updates, rejects aborted/closed operations and ignores obsolete seek results. A compatible stream gets one automatic rebuild at the same position/state. Persistent errors offer original playback below the picture; concrete media codes, fragment indices and timing are available in Output → Image Caption Gallery · Video. Settings → Video → Playback speed offers 0.25×–4×. Changes apply immediately without reloading, and persist for this Gallery across seeking and video switches. Native player speed changes are reflected in Settings too. Playback reads the requested part of the remote file, without first preparing a complete video:
For example, AAC audio is not supported by VS Code's webview, so H.264/AAC MP4 uses compatibility fragments. See VS Code's supported media formats. Convert with FFmpeg also uses short fragments. Play original video lets you try native playback if compatibility processing fails. Seeking into an uncached fragment can still briefly wait for server decoding and transfer. The player offers Original, 2K, 1080p and 720p below the picture. The default is the lower of source resolution and 2K, with no upscaling. 2K means up to 2560 × 1440; 1080p is 1920 × 1080 and 720p is 1280 × 720. Portrait swaps the bounds; other aspect ratios fit proportionally. Use Settings → Video quality or Quality changes retain position, speed, volume and paused/playing state, and preserve Caption drafts. Each Gallery retains its own selection; new galleries use VS Code defaults, without writing workspace settings from UI changes. Legacy Gallery preview defaults migrate to the new configured default; legacy 480/2160 configuration values map to 720p/2K. The header keeps original dimensions and the bottom row shows playback dimensions, marking smaller unchanged sources as Original. Loading, errors and fallback actions sit below the picture; technical details appear in Settings instead of a permanent banner. Image pixel limits and pinch zoom remain image-only. Install FFmpeg and ffprobe on the extension host: your computer for local folders, or the remote server for Remote SSH. Compatibility playback needs
The media endpoint listens only on the extension host's loopback interface, uses revocable random URLs, and is privately forwarded by VS Code under Remote SSH. No manual public port is needed. Playback uses at most two FFmpeg jobs. First-frame extraction has a separate CPU-bounded lane of 1–4 jobs, so another Gallery’s playback cannot starve posters; each poster decoder/filter/encoder uses one thread and duplicate work is shared. Posters try limited input probing, with one normal-probing fallback for unusual containers; they never scan the video to pick a representative scene. Disk cache hits bypass FFmpeg detection, including after restarting the host. Completed fragments and posters use a separate 512 MiB LRU cache, keyed by source path, modification time, size, tool and playback settings. Visible grid posters are protected until released. Original media is never modified. Compatibility playback uses the first video/audio streams and omits embedded subtitles. It does not implement automatic bitrate adaptation. Shortcuts
Dataset layoutThe caption file must use the same base name as the image or video:
Compatibility
PrivacyImage Caption Gallery reads captions when the Text pane is visible and modifies them only when you explicitly save. It streams opened videos through a private loopback endpoint and VS Code SSH forwarding, and caches image previews, video posters and requested compatibility fragments in extension storage on the extension host, leaving original media untouched. It does not upload dataset files to external services and does not include telemetry. Development
Open the repository in VS Code and press Run Roadmap
ContributingIssues and pull requests are welcome. Please keep changes focused and describe how you tested local and Remote SSH behavior. License |