Dataverse Data Sync Utility

Dataverse Data Sync Utility is a VS Code extension that opens a progressive wizard for comparing and migrating data between two Dataverse or Dynamics 365 CE environments.

The extension is designed around relationship-aware migration. It classifies tables from Dataverse metadata, computes a dependency-safe migration order, detects cycles, supports multi-pass migration, and surfaces plan details before any write occurs.

The Problem

Data migration between Dataverse and Dynamics 365 environments is complex and error-prone:

• Relationship dependencies can cause foreign key violations if tables are migrated out of order.
• Cyclic dependencies and N:N relationships require careful planning and multi-pass orchestration.
• Manual tracking of which records were created, updated, skipped, or failed is tedious and error-prone.
• Field defaults differ between environments, leading to data validation errors or inconsistent state.
• No pre-migration visibility — you don't know what will fail until execution starts.

What Makes It Different

• Dependency-aware ordering — automatically detects relationships and computes a safe migration sequence without manual intervention.
• Cycle detection and multi-pass support — handles circular dependencies by running migration passes until all tables succeed.
• Plan before migration — preview exactly what will be created, updated, and skipped before writing any data.
• Field-level metadata awareness — respects system/read-only columns, applies table-specific defaults, and validates data types.
• Comprehensive result tracking — detailed logs for created, updated, skipped, and error records with CSV export.
• Relationship graph visualization — inspect lookup, polymorphic, activity party, and N:N dependencies interactively.

Why VS Code Extension

• Developer-first workflow — integrates into your existing VS Code environment; no additional tools or web portals needed.
• Progressive wizard UX — step-by-step guidance with persisted state, reset options, and confirmations to prevent mistakes.
• Immediate feedback — real-time logging and results visualization without context switching.
• Secure authentication — leverages VS Code's built-in Microsoft authentication with MFA support and device-code fallback.
• Exportable output — results and logs can be exported as CSV for auditing, compliance, and team sharing.

Features

• MFA-safe sign-in using the VS Code Microsoft authentication provider, with MSAL interactive or device-code fallback.
• Progressive multi-step webview wizard with stepper, persisted state, reset actions, and confirmations.
• Source and target environment discovery through Power Platform APIs.
• Table classification using EntityDefinitions metadata.
• Table-aware field defaults for writable vs system/read-only columns.
• Migration plan tabs for summary, missing, different, and relationships.
• Relationship graph analysis for lookup, polymorphic, activity party, and N:N dependencies.
• Multi-pass migration orchestration with deferred state/status handling.
• CSV export for migration plans and error output.
• Output-channel-backed execution logging.

Architecture

The extension uses a clean split between VS Code UI orchestration, Dataverse service access, and pure planning utilities.

• src/extension.ts: command registration and activation.
• src/webview/getWizardHtml.ts: webview HTML, styling, and front-end message bridge.
• src/webview/wizardPanel.ts: panel controller, state persistence, and message handling.
• src/services/authService.ts: Microsoft sign-in, token caching, and fallback auth.
• src/services/environmentService.ts: Power Platform environment discovery.
• src/services/dataverseMetadataService.ts: Dataverse entity, attribute, and relationship metadata access.
• src/services/dataverseDataService.ts: Dataverse Web API requests, paging, and retry logic.
• src/services/planService.ts: table classification, default selections, and diff planning.
• src/services/migrationService.ts: migration execution orchestration.
• src/utils/dependencyGraph.ts: generic graph and cycle detection.
• src/utils/relationshipResolver.ts: dependency resolution for selected tables.
• src/utils/diffEngine.ts: field-level change detection.
• src/utils/csv.ts: CSV export helpers.
• tests/*: unit tests for graph, resolver, diff, and CSV logic.

How To Use

Process Flow

flowchart LR
	A[Install Extension] --> B[Open Command Palette]
	B --> C[Launch Wizard]
	C --> D[Sign In]
	D --> E[Select Source]
	E --> F[Select Target]
	F --> G[Select Tables]
	G --> H[Review Defaults and Plan]
	H --> I[Run Migration]
	I --> J[Review and Export Results]

For a concise business-friendly overview, see SOLUTION_OVERVIEW.md.

Getting Started

1. Install the extension from the VS Code Marketplace or load the .vsix locally.
2. Open VS Code and press Ctrl+Shift+P (Windows/Linux) or Cmd+Shift+P (Mac).
3. Search for and run "Dataverse Data Sync: Open Wizard".
4. Authenticate using your Microsoft account (MFA-safe via device code if needed).
5. Follow the wizard step-by-step to configure your migration.

Step-by-Step Usage

• Step 1 — Login: Authenticate to your tenant using VS Code's built-in Microsoft provider.
• Step 2 — Source Environment: Browse and select the source Dataverse environment to migrate data from.
• Step 3 — Target Environment: Select the destination environment where data will be migrated to.
• Step 4 — Select Tables: Choose which tables to migrate. The extension auto-detects dependencies and relationships.
• Step 5 — Field Defaults: Review system/read-only columns and adjust defaults for each table if needed.
• Step 6 — Build Plan: Let the extension analyze relationships, detect cycles, and compute a safe migration order.
• Step 7 — Confirm Order: Review the proposed migration sequence, warnings, and relationship graph before proceeding.
• Step 8 — Review Results: Monitor real-time execution logs, review created/updated/skipped/error counts, and export CSV reports.

Tips

• Dry run first: Before committing to a full migration, review the plan and test with a subset of tables.
• Check the dependency graph: Understand relationships between tables to avoid unexpected failures.
• Export results: After migration, export logs and results as CSV for auditing and compliance tracking.
• Reset state: Use the reset option in the wizard to clear cached selections and start fresh if needed.

Current Scope And Limitations

• The wizard, planning flow, and migration execution are implemented and runnable, but some Dataverse edge cases still need deeper endpoint-specific handling for production tenants.
• Plain annotation notes can now be copied during execution when enabled in Step 5. File, image, and richer attachment-specific handling still need endpoint-specific hardening.
• Supported activity entities can now deep-insert broader activity-party lists during create when referenced party records were migrated in the same run, and related parties can still be appended after create. Polymorphic lookup targets are now preserved in metadata for better planning, while fully action-driven close semantics for closure-sensitive records still need deeper endpoint-specific handling.
• Bulk APIs such as CreateMultiple, UpdateMultiple, UpsertMultiple, and $batch are not yet negotiated dynamically; the current runtime uses per-record Web API calls with retry.