Speech Workbench

Inspect, play, convert, and transcribe speech audio without leaving Visual Studio Code. Speech Workbench combines a synchronized waveform, spectrogram, RMS energy, and time-aligned experiment data in one practical workspace for speech engineers, researchers, and anyone debugging audio or speech-to-text results.

Open a supported audio file and it becomes an interactive analysis surface. Add a transcript, confidence scores, latency measurements, or any other timestamped CSV/JSON data and inspect it against the exact moment in the recording.

Why Speech Workbench?

• See audio and recognition data together. Every chart, transcript segment, data row, and playhead shares the same time axis.
• Move quickly from recording to result. Record a microphone, watch Azure Speech results arrive, and export the transcript as CSV.
• Compare more than words. Overlay confidence, latency, speaker, locale, and custom experiment measurements from ordinary CSV or JSON files.
• Stay inside VS Code. Browse audio-only resources, convert formats, configure team defaults, and use Azure Speech from a dedicated Activity Bar view.
• Play audio locally. Opening, decoding, visualizing, and converting files do not require Python or an external service.

Quick start

1. Install Speech Workbench from the Visual Studio Marketplace.
2. Open a folder containing audio, then select the Speech Workbench icon in the Activity Bar.
3. Expand My Resources and select an audio file. You can also open a supported file directly from the Explorer.
4. Press Space to play or pause. Click a chart to seek, use Ctrl/Cmd+wheel to zoom, and use Shift+wheel to pan.
5. Select Add data to overlay a timestamped CSV or JSON file.

Supported audio formats: .wav, .mp3, .flac, .ogg, .opus, .aac, and .m4a.

Explore audio

Speech Workbench registers a read-only custom editor for supported audio files. The header shows the file format, sample rate, channel count, duration, and size. For WAV files, the original header sample rate is shown even if the browser audio engine resamples during decoding.

Playback

• Play/pause, stop, and loop controls
• Seek bar with current and total time
• Volume control
• Playback speed from 0.5x to 2x
• Automatic scrolling to keep the playhead in view while zoomed
• Automatic reload when the audio file changes on disk

Keyboard shortcuts:

Visualizations

Toggle any combination of:

• Waveform for amplitude over time
• Spectrogram for frequency energy over time, with configurable FFT size, dB range, and maximum frequency
• RMS energy for a compact view of signal power

All charts use a synchronized playhead and time window. Click a chart to seek, use Ctrl/Cmd+wheel to zoom, and pan with Shift+wheel, the top overview bar, or Alt+drag. Select Reset zoom to return to the full recording. Individual charts and tracks can be collapsed to keep a busy analysis readable.

Open the settings control above the charts to choose visible charts, show or hide the timeline table and live values, configure the spectrogram, and manage data track visibility. Save current view as workspace default writes the current layout to workspace settings for the next file you open.

Overlay time-aligned data

Select Add data in the audio editor and choose one or more .csv or .json files. Speech Workbench turns:

• numeric columns into auto-scaled line charts;
• text and categorical columns into time-segment lanes;
• start/end ranges into highlighted segments;
• the row at the playhead into live values and a synchronized timeline table.

Use the shared column legend to show or hide a column across every track. Track controls let you collapse, reorder, or hide tracks. Hidden tracks can be restored from Settings > Data tracks. The bottom timeline table can be resized, collapsed, and used to seek to the active row.

Companion files

Related data can load automatically when it sits beside the audio file. For example, opening meeting.wav discovers files such as:

meeting.transcription.csv
meeting.transcription.json
meeting.latency.csv
meeting.experiment-a.json

Automatic loading and filename patterns are configurable with speechWorkbench.autoLoadCompanions and speechWorkbench.companionPatterns.

Data format

A data track is a CSV document or JSON array with a time column measured in seconds. For example:

start,end,text,latency_ms,confidence
0.50,1.20,hello,120,0.98
1.25,2.10,world,90,0.95

[
  { "start": 0.5, "end": 1.2, "text": "hello", "latency_ms": 120 },
  { "start": 1.25, "end": 2.1, "text": "world", "latency_ms": 90 }
]

Time columns are detected using speechWorkbench.timeColumnCandidates; common names such as time, timestamp, and start work by default. An optional end column creates duration-based segments. The candidate names can be customized.

Speech Workbench also understands Azure Speech-style JSON objects containing a phrases array. It converts offset_ms and duration_ms into start and end seconds while retaining fields such as text, speaker, and locale.

Transcribe with Azure Speech

The Developer Tools view contains two Azure Speech playgrounds. Both fetch the locales supported by your configured Speech resource when possible and use a built-in locale list as a fallback.

Azure Speech usage may incur charges in your Azure subscription. Audio is sent to the Azure Speech resource you configure only when you start a transcription.

Configure Azure Speech

1. Open Speech Workbench > Developer Tools > Speech to Text.
2. Select Speech Configuration.
3. Choose one authentication method:
   • Azure sign-in: select a subscription and Speech resource using your VS Code Microsoft account. Your account needs access to list the resource and the Cognitive Services Speech User role on it.
   • API key: enter the Speech resource key and region. The key is stored in VS Code SecretStorage.
4. Open Real-time Transcription or Fast Transcription.

The command Speech Workbench: Configure Speech... opens the same setup flow. Configuration can be changed or removed at any time. An API key entered directly in speechWorkbench.azureSpeech.apiKey is supported as a fallback, but SecretStorage is recommended.

Microsoft sign-in with Fast Transcription requires a Speech resource with a custom subdomain. Depending on your Azure permissions and whether local authentication is enabled, Speech Workbench may securely retrieve the resource key for REST operations.

Real-time transcription

Use this mode to watch interim and final recognition results arrive as audio is streamed at real-time speed.

From an audio file

1. Open Real-time Transcription.
2. Choose the recognition options, then select Choose file.
3. Select Start. Progress and elapsed time appear while the file is streamed.
4. Select Stop at any time, or wait for the file to finish.
5. Select Export CSV or Copy when results are available.

From a microphone

1. Install FFmpeg and ensure ffmpeg is on PATH, or set speechWorkbench.recording.ffmpegPath to the executable.
2. Choose a microphone and select Record.
3. Watch the live input meter, elapsed time, interim text, and final segments.
4. Select Stop recording. You can then export the transcript and save the captured recording as a WAV file.

Microphone capture supports Windows through DirectShow, macOS through AVFoundation, and Linux through the default PulseAudio input. The last selected microphone is remembered. FFmpeg is required only for microphone capture; file playback, visualization, conversion, and file-based transcription do not need it.

Real-time options include:

• recognition language and profanity handling;
• Simple or Detailed output;
• phrase lists for names, terminology, and domain-specific words;
• language identification at the start of a session or continuously, with up to 4 or 10 candidate locales respectively;
• speaker diarization, with speaker and detected-language tags per segment;
• raw Speech SDK configuration JSON for advanced SDK properties.

Final segments include timestamps and, when available, speaker, language, and end-of-speech latency tags.

Fast transcription

Use this mode when you want a complete file transcribed in one REST request instead of waiting for real-time playback.

1. Open Fast Transcription and select Choose file.
2. Choose one or more locales and the desired profanity handling.
3. Optionally enable diarization, set the maximum speakers, select audio channels, or add a phrase list.
4. Select Transcribe.
5. Review the combined transcript and timed segments, then select Export CSV.

The Advanced section accepts a raw Fast Transcription definition JSON object. When non-empty, it replaces the generated definition and overrides the form fields. This is useful for service options that are not yet represented in the UI.

Transcript exports

Real-time and Fast Transcription export CSV files that can be loaded back into the audio editor as data tracks. Exports include the available timing, duration, speaker, language, latency, channel, and text fields, which makes a transcription immediately useful for visual inspection beside the source audio.

Convert audio

Open Developer Tools > Audio Tools > Convert Audio Format, or right-click a supported audio file in My Resources or the Explorer and select Convert Audio Format....

The converter can produce:

• WAV as 16-bit PCM;
• MP3 at 96, 128, 160, 192, 256, or 320 kbps.

Keep the source sample rate or convert to 8, 16, 22.05, 44.1, or 48 kHz. Channels can be preserved, mixed to mono, or written as stereo. Select Convert & Save... to choose the destination. Conversion runs locally.

Activity Bar views

My Resources

Shows only supported audio files in the open workspace, nested by folder. Hidden folders and node_modules are excluded. Select a file to open it in Speech Workbench, or use its context menu to convert it. The view refreshes when audio files are created, deleted, renamed, or moved; a manual refresh action is also available.

Developer Tools

Provides direct access to:

• Speech Configuration
• Real-time Transcription
• Fast Transcription
• Convert Audio Format

Settings

Search for Speech Workbench in VS Code Settings. Analysis settings are scoped to the workspace, so teams can commit shared defaults in .vscode/settings.json.

Speech-to-text defaults

Settings under speechWorkbench.stt.* initialize both transcription playgrounds:

Azure resource settings are under speechWorkbench.azureSpeech.*. Prefer the guided Speech Configuration command instead of editing these values manually.

Commands

Open the Command Palette with Ctrl+Shift+P / Cmd+Shift+P and type Speech Workbench:

Workspace trust and privacy

• Audio playback is disabled in untrusted workspaces.
• Local analysis and conversion happen inside the extension/webview and do not require a cloud service.
• Transcription sends the selected or recorded audio to the Azure Speech resource configured by the user.
• API keys entered through the setup command are stored using VS Code SecretStorage. Avoid placing secrets in committed workspace settings.
• Speech Workbench does not require Python.

Troubleshooting

An audio file does not appear in My Resources

Confirm that its extension is supported and refresh the view. Hidden folders and node_modules are intentionally excluded.

The browser cannot decode a supported extension

Actual codec support depends on the media codecs available to the VS Code runtime and operating system. Convert the source to PCM WAV for the most predictable result.

No microphone is listed

Confirm that FFmpeg is installed and available at speechWorkbench.recording.ffmpegPath. On Windows, use a full FFmpeg build with DirectShow and allow desktop apps to access the microphone. On macOS, check microphone permission. On Linux, verify that the default PulseAudio input works.

Azure Speech is not configured

Run Speech Workbench: Configure Speech... and choose Azure sign-in or API key. For sign-in, verify resource access and the Cognitive Services Speech User role. For API-key authentication, verify that the region matches the resource.

Companion data did not load

Check that automatic companion loading is enabled, the filename matches a configured pattern, and the file contains a recognized time column with values in seconds. You can always load the file manually with Add data.