Supercheck for Azure DevOps

Integrate Supercheck uptime monitoring, Playwright browser tests, API tests, and k6 performance tests into your Azure DevOps pipelines.

Features

Pipeline Tasks

Trigger Job — Trigger a Supercheck job with a job-scoped trigger key and optionally wait for completion. Can emit JUnit XML for PublishTestResults@2, publish a rich build summary to the Extensions tab, enrich test output with Supercheck test metadata, and fails fast on invalid keys or timeout/poll values.
Deploy Config — Push supercheck.config.ts monitoring-as-code to your Supercheck instance. Validates the working directory and config path before invoking the CLI.
Validate Config — Validate supercheck.config.ts without deploying. Use in PR builds to catch configuration errors early. Optionally shows a diff of what would change.
Health Check — Verify Supercheck API health and optionally assert a specific monitor is healthy after deployment.
Pause/Resume Monitors — Pause or resume Supercheck monitors during deployments to suppress false alerts. Accepts a comma-separated list of monitor IDs.
Gate Check — Server-side (agentless) gate task for release pre/post-deployment conditions. Verifies Supercheck API health without requiring a pipeline agent.

Supercheck Status — Dashboard widget that displays Supercheck monitor status or latest job run results directly on your Azure DevOps project dashboard. Configure it with a service connection and choose between monitoring a specific monitor or showing the latest run for a job.

Service Connection

A custom service endpoint stores your Supercheck URL and CLI token securely, with built-in "Test Connection" support.

The service connection token is used for authenticated API access such as polling run status and fetching run/test details. The Trigger Job task separately requires a job-scoped trigger key to start a run.

Setup

1. Create a Supercheck CLI Token

In your Supercheck dashboard, go to Project Settings → CLI Tokens
Click Create Token and copy the token (shown only once)

Use a CLI token such as sck_live_* in the Azure DevOps service connection. The extension validates this client-side in the service connection UI and also checks it at runtime when waitForCompletion is enabled.

2. Create or Copy the Job Trigger Key

Open the Supercheck job you want to run from Azure DevOps
Go to that job's CI/CD Settings
Create or copy the job trigger key, for example sck_trigger_*

The trigger key is job-specific and is used only to start the run.

3. Add a Service Connection

In Azure DevOps, go to Project Settings → Service connections
Click New service connection → Supercheck
Enter your Supercheck instance URL and CLI token
Click Verify and save

4. Use in Pipelines

# Gate a deployment on Supercheck tests passing
steps:
  - task: SupercheckTriggerJob@0
    displayName: 'Run E2E tests'
    inputs:
      supercheckConnection: 'My Supercheck Connection'
      jobId: 'your-job-id-here'
      triggerKey: '$(SUPERCHECK_TRIGGER_KEY)'
      waitForCompletion: true
      timeoutSeconds: 300
      failOnTestFailure: true
      publishJUnitResults: true
      junitResultsFile: '$(Common.TestResultsDirectory)/supercheck/TEST-supercheck.xml'

  - task: PublishTestResults@2
    displayName: 'Publish Supercheck test results'
    condition: always()
    inputs:
      testResultsFormat: 'JUnit'
      testResultsFiles: 'TEST-supercheck.xml'
      searchFolder: '$(Common.TestResultsDirectory)/supercheck'
      mergeTestResults: true
      failTaskOnFailedTests: false
      testRunTitle: 'Supercheck'

Trigger Job Requirements

The Trigger Job task uses two different credentials:

triggerKey is required and must start with sck_trigger_ or legacy job_
supercheckConnection is still required for the base URL
when waitForCompletion: true, the service connection token must be a CLI token beginning with sck_live_

This split mirrors the Supercheck API:

trigger key to start the run
CLI token to read run status, run details, and test metadata

JUnit Output And Azure DevOps Tests

When publishJUnitResults is enabled, the Trigger Job task writes a JUnit XML file that can be consumed by PublishTestResults@2.

The generated JUnit includes:

a synthetic gate testcase for the overall run
detailed Supercheck testcases when the run API returns testResults
suite properties such as run ID, job ID, status, duration, and report URL
enriched testcase metadata from Supercheck test definitions

Tag Conventions For Enriched Test Results

Supercheck tags only allow letters, numbers, underscores, and hyphens. To enrich Azure DevOps test output, use tags like these on your Supercheck tests:

owner-performance
service-cabmd15
area-loadgate
priority-high

The Trigger Job task maps these into JUnit like this:

owner-* or owner_* -> Azure DevOps test Owner
service-*, area-*, and priority-* -> JUnit classname so the test row is easier to group and filter
test description, priority, and tags -> JUnit system-out, visible in the Azure DevOps test details/logs view

Example metadata to validate the enrichment:

Description: Load gate for Cab15 release validation
Tags:
- owner-performance
- service-cabmd15
- area-loadgate
- priority-high

With that metadata, the imported test result should show:

Owner: performance
Test file/class grouping similar to cabmd15/loadgate/high
Console output containing the description and tags

# Deploy monitoring-as-code config
steps:
  - task: SupercheckDeployConfig@0
    displayName: 'Deploy monitoring config'
    inputs:
      supercheckConnection: 'My Supercheck Connection'
      showDiff: true
      dryRun: false

# Post-deployment health check
steps:
  - task: SupercheckHealthCheck@0
    displayName: 'Verify monitors healthy'
    inputs:
      supercheckConnection: 'My Supercheck Connection'
      checkMonitor: true
      monitorId: 'your-monitor-id'
      waitForMonitorSeconds: 60

# Validate config in a PR build (read-only, no deploy)
steps:
  - task: SupercheckValidateConfig@0
    displayName: 'Validate monitoring config'
    inputs:
      supercheckConnection: 'My Supercheck Connection'
      showDiff: true

# Pause monitors before deployment, resume after
steps:
  - task: SupercheckPauseMonitors@0
    displayName: 'Pause monitors for deploy'
    inputs:
      supercheckConnection: 'My Supercheck Connection'
      action: pause
      monitorIds: 'monitor-id-1,monitor-id-2'

  - script: echo "Deploy your app here"

  - task: SupercheckPauseMonitors@0
    displayName: 'Resume monitors after deploy'
    condition: always()
    inputs:
      supercheckConnection: 'My Supercheck Connection'
      action: resume
      monitorIds: 'monitor-id-1,monitor-id-2'

Full Pipeline Example

trigger:
  branches:
    include: [main]

stages:
  - stage: Test
    jobs:
      - job: SupercheckE2E
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: SupercheckTriggerJob@0
            displayName: 'Run Supercheck E2E tests'
            inputs:
              supercheckConnection: 'Supercheck Prod'
              jobId: '$(SUPERCHECK_E2E_JOB_ID)'
              triggerKey: '$(SUPERCHECK_E2E_TRIGGER_KEY)'
              waitForCompletion: true
              timeoutSeconds: 600
              failOnTestFailure: true
              publishJUnitResults: true
              junitResultsFile: '$(Common.TestResultsDirectory)/supercheck/TEST-supercheck.xml'
          - task: PublishTestResults@2
            displayName: 'Publish Supercheck test results'
            condition: always()
            inputs:
              testResultsFormat: 'JUnit'
              testResultsFiles: 'TEST-supercheck.xml'
              searchFolder: '$(Common.TestResultsDirectory)/supercheck'
              mergeTestResults: true
              failTaskOnFailedTests: false
              testRunTitle: 'Supercheck E2E'

  - stage: Deploy
    dependsOn: Test
    condition: succeeded()
    jobs:
      - deployment: DeployApp
        environment: production
        strategy:
          runOnce:
            preDeploy:
              steps:
                - task: SupercheckPauseMonitors@0
                  displayName: 'Pause monitors for deploy'
                  inputs:
                    supercheckConnection: 'Supercheck Prod'
                    action: pause
                    monitorIds: '$(SUPERCHECK_PROD_MONITOR_ID)'

            deploy:
              steps:
                - script: echo "Deploy your app here"

            postRouteTraffic:
              steps:
                - task: SupercheckPauseMonitors@0
                  displayName: 'Resume monitors'
                  condition: always()
                  inputs:
                    supercheckConnection: 'Supercheck Prod'
                    action: resume
                    monitorIds: '$(SUPERCHECK_PROD_MONITOR_ID)'

                - task: SupercheckHealthCheck@0
                  displayName: 'Post-deploy health check'
                  inputs:
                    supercheckConnection: 'Supercheck Prod'
                    checkMonitor: true
                    monitorId: '$(SUPERCHECK_PROD_MONITOR_ID)'
                    waitForMonitorSeconds: 30

  - stage: UpdateMonitoring
    dependsOn: Deploy
    condition: succeeded()
    jobs:
      - job: ValidateConfig
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: SupercheckValidateConfig@0
            displayName: 'Validate monitoring config'
            inputs:
              supercheckConnection: 'Supercheck Prod'
              showDiff: true

      - job: DeployConfig
        dependsOn: ValidateConfig
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: SupercheckDeployConfig@0
            displayName: 'Sync monitoring config'
            inputs:
              supercheckConnection: 'Supercheck Prod'
              showDiff: true

Release Gate Usage

The Gate Check task runs as a server-side (agentless) gate in release pipelines. Add it as a pre-deployment or post-deployment gate:

Open your release pipeline and select a stage
Click the pre-deployment or post-deployment conditions icon
Enable Gates and click + Add
Select Supercheck - Gate Check
Configure the Supercheck service connection

The gate evaluates the Supercheck API health endpoint and succeeds when the status is ok.

Output Variables

The Trigger Job task sets these pipeline variables when outputRunId is enabled:

Variable	Description
`SUPERCHECK_RUN_ID`	The Supercheck run ID
`SUPERCHECK_RUN_STATUS`	Final run status (passed/failed/error/blocked)
`SUPERCHECK_RUN_URL`	Direct link to the run in the Supercheck dashboard
`SUPERCHECK_JUNIT_RESULTS_FILE`	Absolute path to the generated JUnit XML file when `publishJUnitResults` is enabled

The Pause/Resume Monitors task sets:

Variable	Description
`SUPERCHECK_PAUSED_MONITORS`	Comma-separated list of monitor IDs that were successfully paused
`SUPERCHECK_RESUMED_MONITORS`	Comma-separated list of monitor IDs that were successfully resumed

Self-Hosted Supercheck

If running a self-hosted Supercheck instance, set the URL in your service connection to your instance's address (e.g., https://supercheck.yourdomain.com).

Development

npm install
npm run bump:patch
npm test
npm run build
npm run package

The root scripts now handle TypeScript compilation, trigger-job test compilation, task dependency installation, and VSIX packaging on both Windows and Linux agents.

Supercheck for Azure DevOps

Clinical Support Systems

Supercheck for Azure DevOps

Features

Pipeline Tasks

Dashboard Widget

Service Connection

Setup

1. Create a Supercheck CLI Token

2. Create or Copy the Job Trigger Key

3. Add a Service Connection

4. Use in Pipelines

Trigger Job Requirements

JUnit Output And Azure DevOps Tests

Tag Conventions For Enriched Test Results

Full Pipeline Example

Release Gate Usage

Output Variables

Self-Hosted Supercheck

Links

Development