Seedbed

Seedbed is a local VS Code extension that profiles a schema or example dataset, uses a model already available through GitHub Copilot Chat to infer semantic rules, and generates validated synthetic records in bounded batches.

Copilot Chat is not treated as an HTTP endpoint. The extension uses the stable VS Code Language Model API (vscode.lm) and the user's existing Copilot sign-in and model access.

Pipeline

1. Input file — select one local JSON, JSON Schema, JSONL, CSV, TSV, YAML, XML, or canonical MSPDI file. Unsupported extensions, remote URIs, oversized files, unsafe XML declarations, and malformed content are rejected.
2. Format parser — the dedicated local parser for that extension identifies the record collection, schema, nesting, field types, required fields, formats, constraints, and observed structure. MSPDI is profiled both as a private complete XML blueprint and as a versioned 12-field semantic leaf-task contract.
3. Automatic LLM analysis — immediately after parsing and model selection, token sizing and one Copilot analysis request start together. The analyzer identifies the domain, entity/data type, categories, realistic ranges and distributions, and cross-field relationships. For MSPDI it also creates a structured production-style vocabulary for the organization, project code, email domain, resources, roles, teams, calendars, workstreams, workflow columns, iterations, views, filters, groups, tables, custom fields and values, locations, and deliverables. Its reusable generation description and vocabulary are cached by source, model, and example-sharing mode and sent unchanged with every later generation request.
4. LLM call tuning — configure the model, total records, records per call, concurrent calls, additional generation requests, output encoding, destination, and optional example sharing. Changing total or per-call records recalculates batches, safe limits, run calls, effective concurrency, and parallel waves.
5. Multi-turn generation and temporary state — MSPDI first receives bounded chronological planning calls. A bounded worker pool then generates and locally validates batches. Every completed root batch is written atomically to its own temporary JSON artifact; sequence.json records sequence, global start index, count, state, and file name even when calls complete out of order.
6. Stitch, parse, and commit — after every manifest entry is ready, the stitcher rereads temporary artifacts in sequence and writes one atomic partial output. A format-specific parser independently reparses those exact bytes against the source schema and requested record count. MSPDI also validates project semantics and references. Only parser-approved output replaces the destination; temporary files are deleted after success, cancellation, or failure.

For MSPDI, the model never reads or writes the 100+ field Project XML task representation. Local code converts XML to compact JSON, Copilot returns task records in that exact same JSON shape, and local code validates the records. The complete parsed XML stays local as a separate blueprint and is never added to the model prompt. At finalization, generated semantic values are mapped into the original leaf-task slots, summary values are recalculated, identities are regenerated, and task/resource/calendar/assignment references are remapped without adding or removing fields.

The analyzer's domain vocabulary also drives every non-task text surface retained by the full blueprint. Project metadata, resource and team names, calendars, workflow columns, sprints/iterations, views, filters, groups, tables, custom fields and values, locations, emails, accounts, codes, hyperlinks, and deliverables use coherent real-world terminology from the selected domain. Placeholder-like analyzer values are rejected and replaced with a plausible operational vocabulary; final MSPDI validation blocks explicit Synthetic, Demo, Example, Placeholder, Dummy, and lorem-style text before commit.

JSON Lines and generic outputs stream incrementally during stitching. Source-shaped MSPDI output is assembled in memory at finalization because the complete cross-referenced tree must be transformed atomically; input size remains subject to the configured profiling limit.

Supported Inputs

Output can be JSON Lines, a JSON array, CSV, generic XML, or MSPDI. MSPDI output is offered only for a canonical MSPDI input. Direct MSPDI generation preserves the complete source schema: project fields, task and summary hierarchy, calendars, resources, assignments, predecessor links, baselines, custom fields, timephased data, board columns, sprints, attributes, and all other parsed node templates retain their field presence and order. Project/task/resource/calendar/assignment GUIDs and identifiers are regenerated, references are remapped, content-bearing text is replaced, dates are shifted, generated leaf semantics are applied, and summary values are recalculated.

The total leaf-task count remains fully editable. At the source leaf count, ordered element/attribute topology and collection cardinalities match exactly. Below that count, the renderer keeps the project root and truncates after the requested number of source-shaped leaves, retaining required summary ancestors and dependent assignments. Above it, the renderer repeats complete hierarchy segments plus a final partial segment and clones matching assignments. IDs, outline values, predecessor links, and assignment references are rebuilt for the resized graph.

MSPDI Compact JSON Contract

The versioned mspdi-task-v1 envelope preserves the local reconstruction schema and sanitized project template once, followed by compact tasks containing:

• taskName and phase
• startDate and finishDate
• percentComplete, milestone, and priority
• estimatedWorkHours and fixedCost
• nullable baseline start and finish dates
• useful task notes

Summary tasks are used locally to derive phase names. Only usable leaf tasks become model generation records; summary tasks and all non-task collections stay in the private full-tree blueprint. The exact-fidelity renderer applies generated names, dates, duration, work, cost, progress, baseline, and notes to existing leaf fields, recalculates existing summary fields, regenerates GUIDs, and remaps references locally.

The local converter can also be run independently:

# Build once in a source checkout; packaged copies already contain this bundle.
npm run compile

# MSPDI XML -> compact, one-line generation JSON
npm run mspdi:json -- to-json input.xml generation.json

# The same validated JSON -> canonical MSPDI XML
npm run mspdi:json -- to-mspdi generation.json generated-project.xml

The JSON-to-MSPDI command rejects generic names such as Plan 1 and Task 2, inconsistent date ranges, invalid milestones, zero-work ordinary tasks, broken baseline pairs, unexpected fields, and type/limit violations.

On Commercial Construction Project with Baselines.xml, this conversion measured:

• 30.2 MB source XML to 845 KB compact JSON, a 97.2% reduction
• 88 previously exposed task fields to 12 semantic fields
• 2,320 XML task elements to 2,016 relevant leaf-task generation records
• approximately 128 tokens per compact task using a conservative character-based estimate
• 200 recommended records per call and 12 planned model calls with the observed 922K-context model

Run Locally

Prerequisites:

• VS Code 1.90 or later
• GitHub Copilot Chat installed, signed in, and entitled to at least one chat model
• Node.js 20 or later

cd tools/SyntheticDataGenerator
npm install
npm test
npm start

npm start builds the extension, opens an Extension Development Host, and opens the generator workbench. You can also press F5 in the tool folder; development hosts open the workbench automatically. In an installed copy, run Seedbed: Open Generator from the Command Palette. The included examples/employee.schema.json is a small smoke-test input.

The command opens a responsive workbench with:

• Native source and destination file pickers
• One-line animated status for file inspection, parsing, automatic LLM analysis, generation, stitching, and final validation
• Source profile summary with format, record count, field count, selected record path, and local parser
• Detailed schema field table plus compact notation showing the outer path and the record object generated in bulk
• Source token estimate and selected-model measurements for the structural prompt, optional examples, and generated record size
• Available Copilot model selector
• An explicit total-record question and a synchronized per-call slider/number control bounded to the safe 1..N range for the selected model
• An editable MSPDI leaf-task count; the source count is the default, while smaller and larger values scale hierarchy segments and dependent collections
• An Auto, 1, 2, or 4 concurrent-call control; Auto uses 3 workers for generic formats and 2 after MSPDI planning
• JSONL, JSON, CSV, XML, and conditional MSPDI output modes
• Explicit schema-only or schema-plus-available-examples privacy selection for ordinary formats; unavailable examples are disabled and MSPDI is always schema-only
• Optional domain and distribution requirements
• Realism-focused semantic planning that uses examples as style and distribution cues without copying unique source values
• Domain-aware synthesis for project metadata, resources, calendars, workflow states, iterations, custom fields, and retained XML text fields
• An LLM exchange audit window showing the exact request and raw response for every analysis, MSPDI planning, generation, retry, and adaptive-split call, with model, scope, attempt, status, timing, copy, and clear controls
• Live capacity, realism-plan, generation, validated-write, finalization, and completion progress with cancellation
• Parser-validated completion count and final file size

The run summary remains visible while configuring the dataset. Closing the panel cancels an active run and removes partial output.

Settings

Batch size has an absolute cap of 200, but the workbench lowers the effective maximum when the selected model, schema width, widest locally observed record, or requested total requires it. Input-context capacity does not imply equivalent output capacity, so structured responses use an independent 16K-token ceiling before safety headroom. The extension host enforces the same analyzed range as the slider. Smaller batches improve JSON reliability; larger batches reduce calls.

If a model still returns fewer records than requested, the generator does not discard the run. It reports the actual entry and object counts, halves that model call into contiguous global-index ranges, validates each smaller result independently, and combines them in order. Malformed responses receive the configured normal retries before the same adaptive split is attempted.

Parallel calls share one semantic plan and one bounded uniqueness registry. A sliding scheduling window limits both active calls and out-of-order buffered batches to the selected concurrency, while one writer commits completed batches in source order. A failed worker stops new scheduling; in-flight calls settle before the partial output is removed. MSPDI workers additionally receive authoritative phase/date plans, validate every task against their assigned window, and are combined in chronological root-batch order.

Validated batches are not held until the final writer catches up. Each root batch is persisted under a .synthetic-batches-* workspace beside the destination, and its atomic sequence manifest is updated after every call. Stitching validates each artifact's sequence and record count before reading it. Normal success, cancellation, validation failure, and generation failure recursively remove that workspace.

Changing total records or records per call immediately recalculates generation batches, model-call count, effective concurrent workers, and parallel call waves. MSPDI schedule planning adds one call per 64 generation batches, so its overhead remains bounded for large requests.

The workbench recalculates capacity whenever the source, Copilot model, or applicable example-sharing mode changes. It uses the selected model's tokenizer to measure representative records locally and the structural prompt, reserves context and output headroom, and defaults total records to the input dataset's scale. This avoids spending another LLM completion call on a capacity guess. A total explicitly entered by the user is preserved across later model or privacy-mode changes, while the per-call value is clamped to the newly analyzed safe range.

Data And Safety Boundaries

• The full source file and source-shaped MSPDI blueprint stay local. MSPDI always sends only the structural 12-field schema and explicit constraints; source task names, phases, dates, notes, resources, assignments, and custom values remain local. Other formats may optionally send the first three bounded example records. Sample-derived literals and extrema are never embedded in the schema payload.
• The LLM audit captures only prompt text already sent to Copilot and response text already returned. It is held in extension-host memory for the panel lifetime, retains at most 50 complete calls or 8 MiB, evicts the oldest settled calls without truncating exchanges, and can be cleared at any time.
• XML containing DOCTYPE or ENTITY declarations is rejected before parsing.
• Model output cannot add undeclared fields and must pass recursive type, required-field, declared-enum, format, pattern, and declared-limit validation. Categories merely observed in examples remain guidance rather than hard limits.
• MSPDI task names are checked case-insensitively across batches with a bounded 64-bit fingerprint index, so ordinary Bloom false positives do not rename tasks. Other top-level identifier fields inferred to be unique use a bounded Bloom filter; it never misses duplicates, while a rare false positive can cause a conservative retry.
• MSPDI project titles and task names must be concrete and natural. Generic numbered placeholders are rejected and regenerated automatically.
• Repeated MSPDI task names and missing, generic, or restated notes are repaired locally after each model response. The repair adds a phase/date/work-package qualifier to a duplicate name and a deliverable/acceptance note without changing dates, progress, work, cost, milestone, priority, or baseline values. This avoids regenerating an otherwise valid large batch.
• Valid MSPDI task dates that land on Saturday or Sunday are normalized locally before validation, preferring the following Monday when it remains inside the planned batch window and otherwise using the preceding Friday. Milestone start/finish dates remain equal, finish dates cannot precede starts, and baseline weekends are normalized independently because a baseline may precede the task window. Ordinary out-of-window weekdays and invalid date values still fail validation and retry.
• Cancellation and failures remove the partial output. Replacing an existing destination uses a rollback backup until final rename succeeds.
• Final JSON and JSONL validation is streaming; CSV uses the CSV parser stream; generic XML uses SAX parsing; MSPDI is reparsed through the strict project adapter and reference checker. A failed final parse leaves any previous destination untouched.
• CSV strings beginning with spreadsheet formula characters are prefixed with an apostrophe.

This is a synthetic-data generator, not a formal anonymization or differential-privacy system. Do not select examples containing secrets, credentials, or data that policy prohibits sending to Copilot. Review generated data before using it in tests or demos.

Current Limits

• One record collection is generated per run. For generic hierarchical files, the largest repeated collection is selected.
• Inputs are profiled in memory and hard-capped at 128 MB; use a representative sample for larger source files. Source-shaped MSPDI output also holds the cloned/resized parsed tree and rendered XML during atomic finalization.
• JSON Schema must describe object records or an array of object records. $ref, oneOf, conditional schemas, and custom vocabularies are not resolved yet.
• MSPDI output at a count different from the source cannot have identical collection cardinality; it instead preserves the source node schemas and repeats or truncates complete hierarchy patterns. Prefixed or noncanonical Project XML is treated as generic XML input and does not enable MSPDI output.
• Cross-record semantics are model-guided; deterministic validation currently covers schema constraints and uniqueness, not arbitrary domain equations.
• Model availability, rate limits, and quota are controlled by the signed-in GitHub Copilot account.
• Copilot service throttling and quota still apply. Use 1 when diagnosing provider limits; Auto is the recommended default.

Validation

npm test

The suite covers format profiling, MSPDI detection, the compact XML→JSON→XML CLI round trip, useful-name and schedule validation, XXE rejection, recursive schema inference, model-aware recommendations, hard versus observed limits, model-response repair, cross-batch uniqueness, exact model-exchange auditing, untrusted webview request validation, all output writers, and cancellation cleanup. It also syntax-checks the browser-side webview script.