Contica Test Tool

Generates negative tests for Contica integrations, and runs them against the live Azure resource — without leaving VS Code.

Integration testing usually proves the happy path: a well-formed message goes in, the expected result comes out. What breaks in production is everything else — a missing required field, a code outside the allowed enum, a value too long for the target schema. Writing those tests by hand means reading the schema, the mapping and the workflow, then hand-crafting a broken message for each rule. That is slow, and it is the work people skip.

This extension does it for you. It reads an integration's real artifacts, works out what should fail and why, has you approve the plan, builds the broken messages, sends them to the Logic App, and tells you whether each one failed the way it was supposed to.

Who it's for: testers and integration engineers working on Contica integrations. You do not need to be a developer to use it.

Before you start — what you need

Three things, and it will not work without any of them.

1. VS Code 1.95 or later.

2. GitHub Copilot Chat, signed in, with a Claude model available on your account. The Analyse and Generate steps call a language model through VS Code's language-model API, which Copilot provides. Without it, those two steps cannot run — you can still browse bundles, switch environments and read existing packs. VS Code offers to install Copilot Chat alongside this extension.

3. A bundle — the integration's artifacts, laid out in a folder. This is the input the whole tool runs on, and it is the part people get stuck on. See Preparing a bundle below. In short: a folder per integration, a folder per entity inside it, and a manifest.json in each that declares which file is the schema, which is the mapping, and which is a real example message.

To run the Test phase as well, you also need Azure access: permission to read the Logic App and invoke its trigger, in the subscription that hosts the integration.

The four phases

Phase	What happens	Who does it
Fetch	The integration's schemas, mappings, examples and workflow are collected into a bundle folder.	You, by hand. There is no automated fetcher.
Analyse	The bundle is read, constraints are counted, and a language model proposes a test plan. Every claim it makes is checked back against the bundle.	The extension, then you review
Generate	The approved plan becomes real broken messages — one file per scenario, with a written explanation of what it breaks and what should happen.	The extension
Test	Each message is sent to the live Logic App, the run history is polled, and every scenario is graded pass, fail or error.	The extension

Garbage in, garbage out. A bundle with no examples and no mappings produces a thin plan full of gaps. A complete bundle produces a rich, grounded one. The quality of everything downstream is set at the Fetch phase.

How it works

  BUNDLE REPO (git)                 One clone per environment - Test / QA / Prod -
  ┌──────────────────┐              so the same integration can differ per stage
  │ INT0001/         │              without anything being merged between them.
  │   manifest.json  │
  │   schemas/       │
  │   mappings/      │
  │   examples/      │
  │   code/          │
  └────────┬─────────┘
           │ read
           ▼
  ┌────────────────────────────────────────────────────────────┐
  │ ANALYSE                                                    │
  │                                                            │
  │  Deterministic pass  counts required fields, maxLength,    │
  │                      patterns and enums straight from the  │
  │                      schemas. No AI involved.              │
  │                                                            │
  │  Reasoning pass ───► Claude, via Copilot                   │
  │                 ◄─── proposed scenarios + gaps             │
  │                                                            │
  │  Grounding check     every AI claim is matched back to a   │
  │                      real line in the bundle. Anything it  │
  │                      cannot prove becomes a gap, not a     │
  │                      test.                                 │
  └───────────────────────────┬────────────────────────────────┘
                              │
                              ▼
                ┌──────────────────────────┐
                │  YOU REVIEW              │  Summary - what was found
                │                          │  Plan    - toggle, delete, add
                │  Nothing proceeds until  │  Gaps    - resolve each one
                │  you approve the plan.   │
                └────────────┬─────────────┘
                             │ approved
                             ▼
  ┌────────────────────────────────────────────────────────────┐
  │ GENERATE                                                   │
  │                                                            │
  │  Mutator             applies each scenario's mutation to a │
  │                      real example message                  │
  │                                                            │
  │  Instructions  ────► Claude, via Copilot                   │
  │                ◄──── how to run it, what to expect         │
  │                                                            │
  │  Output              one broken message + one scenario doc │
  │                      per test, plus pack.json              │
  └───────────────────────────┬────────────────────────────────┘
                              │
                              ▼
  ┌────────────────────────────────────────────────────────────┐
  │ TEST                                         live Azure    │
  │                                                            │
  │  Submit              each message to the Logic App trigger │
  │  Poll                the workflow's run history            │
  │  Grade               pass / fail / error against the       │
  │                      expected outcome from the plan        │
  └───────────────────────────┬────────────────────────────────┘
                              │
                              ▼
                test-results/ in the bundle repo,
                one timestamped folder per run

Analyse and Generate write their output back into the environment's clone of the bundle repo, so a colleague who syncs the repo sees the same plan, the same pack and the same results you do.

Getting started

1. Install. From the Marketplace, or sideload a build:

code --install-extension contica-test-tool-<version>.vsix

2. Point it at bundles. Either set conticaTestTool.bundleDir to a folder containing your integration folders, or use Add / Edit Environment in the Integrations sidebar to clone a bundle repository.

3. Open the Contica Test Tool icon in the Activity Bar. The Integrations sidebar lists every integration in the active environment. An entity with a valid bundle is ready to work on; one missing a manifest.json shows greyed out, and tells you what it is missing.

4. Click an entity, then Analyse. The bundle is read, the constraints are counted, and a plan comes back in three tabs:

Summary — what was found in the bundle, and how much of it.
Plan — the proposed scenarios. Toggle any off, delete them, or write your own.
Gaps — anything the tool could not prove from the bundle. Each one explains what is missing and why it matters. Resolve them by confirming with a quote from the artifacts, marking them not applicable, or deferring.

Nothing generates until you press Approve Plan for Generation. The plan is yours, not the tool's.

5. Generate Test Pack. You get one broken message per scenario, each with a written explanation of what it breaks and what the integration should do about it, plus a pack.json describing the set.

6. Run Test (optional). Pick the Azure resource the pack targets, and the extension submits every message, polls the run history and grades each scenario.

Using it from Copilot Chat

The extension registers a @tester participant:

@tester /analyse INT0001.o009
@tester /generate INT0001.o009

Results land in the same tabs and files as the sidebar. The chat path uses Copilot's own model picker, which lets you pick a model that does not consume premium quota. For the sidebar path, use Contica Test Tool: Choose AI Model.

Preparing a bundle

A bundle is a folder of an integration's real artifacts, plus manifest.json files that say what each one is. The folder names below are convention — the manifest declares the actual paths, so the loader does not depend on them.

INT0001/                       integration ID, uppercase
├── manifest.json              declares the shared files
├── schemas/                   canonical model, shared libraries
├── mappings/                  integration-wide mappings
├── standards/                 error handling guides
├── code/                      shared code, connection config
│
└── INT0001.i001/              one folder per entity
    ├── manifest.json          declares this entity's files
    ├── examples/              at least one real happy-path message
    ├── schemas/               entity-specific schemas
    ├── mappings/              this entity's mapping
    ├── documentation/         exported Confluence docs
    └── code/                  workflow.json, custom code

Entity IDs are INT####.i001 for inbound or INT####.o005 for outbound — lowercase letter, three digits, optionally a trailing T. Outbound entities often have no examples of their own; that is expected, the source message lives at the upstream entity.

The one file you cannot skip

At least one real happy-path example message per entity. Every negative test is made by breaking a real message, so with no example there is nothing to break, and Generate stops with a gap that says exactly that. Capture one from a Service Bus trace, a Logic App run input, or a recorded test payload.

What each folder buys you

Folder	What it unlocks	Needed?
`examples/`	Everything. It is the baseline every test is made from.	Required — one per entity
`schemas/`	Required-field, maxLength, pattern and enum scenarios	Strongly recommended
`mappings/`	Mapping-correctness scenarios	Strongly recommended
`documentation/`	Business-rule scenarios	Optional, high value
`code/`	Workflow, retry and routing scenarios	Optional
`standards/`	Retry and error-handling scenarios	Optional

A minimal bundle

INT0001/INT0001.i001/manifest.json:

{
  "manifestVersion": 1,
  "scope": "entity",
  "integrationId": "INT0001",
  "entityId": "i001",
  "entityName": "Incoming product - Infor M3",
  "direction": "inbound",
  "sourceSystem": "Infor M3",
  "format": "xml",
  "inheritsFrom": "../manifest.json",
  "entityFiles": {
    "examples": [
      { "path": "examples/INT0001.i001 Item Master.xml", "purpose": "happy-path" }
    ]
  }
}

And INT0001/manifest.json alongside it:

{
  "manifestVersion": 1,
  "scope": "integration",
  "integrationId": "INT0001",
  "integrationName": "Product",
  "sharedFiles": {},
  "entities": ["i001"]
}

That is a complete, valid, minimal bundle — Analyse and Generate will run against it. Add a source schema ("purpose": "source"), then a mapping ("purpose": "source-to-canonical"), then documentation, re-running Analyse each time: the plan grows and the gap list shrinks as the bundle gets more complete.

Two things that trip people up: include the entity ID in example and mapping filenames so the loader matches them to the right entity, and declare "encoding" for any UTF-16 schema (utf-16-le, utf-16-be, windows-1252) or convert it to UTF-8 first.

Environments

An environment is a named bundle-repo profile — Test, QA, Prod — and each one is its own git clone. Artifacts can differ per stage without anything being merged between them. Add / Edit Environment creates one, Switch Environment activates it, and the sidebar plus every subsequent run reads from the new clone immediately.

Settings

All under conticaTestTool.* in the Settings UI or settings.json.

Setting	What it does
`bundleDir`	Folder containing integration bundles. Ignored while a bundle-repo environment is active.
`bundleRepo.url`	Git remote for the shared bundle repository, HTTPS or SSH.
`bundleRepo.localPath`	Where that repo is cloned. Set for you by Add / Edit Environment.
`bundleRepo.branch`	Branch to track. Defaults to `main`.
`environments`	The registered environments. Managed by the Add and Switch commands.
`activeEnvironment`	Which one is active. Set by Switch Environment.

How a verdict is decided

Each scenario is graded against the expected outcome recorded in the plan, using the workflow run itself rather than just the trigger response — a Logic App returns 202 long before it has finished, so the response alone would call a later failure a pass.

The run belonging to a scenario is identified by the run id the trigger returns, or, for Service Bus and Blob triggers which return none, by excluding the runs that already existed before the message was sent. When neither is available the scenario is reported as unverified rather than being matched on timing alone, because a verdict borrowed from the scenario before it is indistinguishable from a real one.

A negative scenario that fails at a different action than the plan expected is a fail, not a pass. Failing for the wrong reason is still a regression.

Support

Licensed under a proprietary end user licence: free to install and use, source not published. Report problems through your Contica contact.