Virtual Device
A high-performance, device-agnostic streaming and control platform designed for both browser environments and VS Code Extension Webviews.
Tech Stack
- Backend: Fastify with Pino logging,
@fastify/websocket, @grpc/grpc-js, shared memory
- Frontend: SolidJS, Vite, TailwindCSS v4, DaisyUI v5
- Tooling: Biome (
@biomejs/biome)
The Android emulator provider opens streamScreenshot over gRPC and prefers
RGBA delivery through a shared memory file, with standard gRPC payloads as a
fallback. The backend snapshots only the frame selected for delivery, sends
metadata followed by a binary WebSocket message, and keeps at most the newest
pending frame while waiting for the browser's paint acknowledgement. This
bounds memory use and latency when the producer is faster than the UI.
Fastify's ws transport runs with compression disabled, bounded inbound
messages, and a frame-acknowledgement timeout. Replacing it with a different
server would add integration and shutdown complexity without improving the
single-client localhost bottleneck.
DeviceCanvas sizes capture to the visible viewport, pauses the stream while
the page is hidden, and uploads frames directly to a reusable WebGL texture.
Pointer movement is reduced to the latest sample per animation frame so stale
positions cannot accumulate, while down, up, cancel, and boundary-release
events remain immediate. Input travels back over the same WebSocket and through
a cached gRPC streamInputEvent writer. Coordinate rotation and release at the
display boundary match Android Studio's emulator input behavior.
Toolbar keys, pointer input, horizontal and vertical wheel scrolling, and
Ctrl+wheel two-finger pinch gestures use the same WebSocket. Shift+wheel
scrolls horizontally. The HTTP key and touch routes remain supported for
compatibility and automation.
The main flow is split by responsibility:
server/providers/android-emulator/: emulator process, gRPC, and shared
memory lifecycle.server/routes/video-stream.ts: WebSocket validation, frame backpressure,
and input forwarding.src/components/viewer/DeviceCanvas.tsx: connection, paint scheduling, and
pointer lifecycle.src/components/viewer/webglRenderer.ts: reusable RGBA WebGL renderer.
Resizing restarts only the emulator capture stream while keeping the WebSocket
connected. Hiding the page pauses capture; closing the viewer, stopping a
device, or shutting down the server releases the stream and active touches.
Future remote streaming
WebRTC with hardware video encoding is intentionally out of scope for the
local path, where encoding would add complexity and latency. It remains the
preferred future transport if the viewer is exposed over a network and
bandwidth or congestion control becomes more important than raw-frame latency.
Getting Started
VS Code Extension
Build the installable extension:
pnpm package:extension
Install virtual-device.vsix, then select the Virtual Device icon in the Activity Bar.
The device manager and viewer open in the sidebar. You can also run
Virtual Device: Open Virtual Device from the Command Palette.
Use Browser in a running device's header to open its local URL in a new browser tab.
Each URL selects that device, so you can keep several emulators open in separate
tabs. Browser tabs stay available while the VS Code extension is running.
Development Mode
Runs the Fastify backend and Vite frontend concurrently with clean exit lifecycle management:
pnpm dev
- Frontend:
http://localhost:5173
- Backend API:
http://localhost:3000
pnpm check
Production Build
pnpm build
pnpm start
The build type-checks the project and emits one deployable package to dist/.
Its bundled Fastify server serves both the API and optimized browser client.
Run pnpm install --prod inside dist/ when deploying it separately. The server
binds to 127.0.0.1:3000 by default; HOST, PORT, and LOG_LEVEL can override
those values.
