Supercheck for Azure DevOps

Integrate Supercheck uptime monitoring, Playwright browser tests, API tests, and k6 performance tests into your Azure DevOps pipelines.

Features

Pipeline Tasks

• Trigger Job — Trigger a Supercheck job with a job-scoped trigger key and optionally wait for completion. Can emit JUnit XML for PublishTestResults@2, enrich test output with Supercheck test metadata, and fails fast on invalid keys or timeout/poll values.
• Deploy Config — Push supercheck.config.ts monitoring-as-code to your Supercheck instance. Validates the working directory and config path before invoking the CLI.
• Health Check — Verify Supercheck API health and optionally assert a specific monitor is healthy after deployment.

Service Connection

A custom service endpoint stores your Supercheck URL and CLI token securely, with built-in "Test Connection" support.

The service connection token is used for authenticated API access such as polling run status and fetching run/test details. The Trigger Job task separately requires a job-scoped trigger key to start a run.

Setup

1. Create a Supercheck CLI Token

1. In your Supercheck dashboard, go to Project Settings → CLI Tokens
2. Click Create Token and copy the token (shown only once)

Use a CLI token such as sck_live_* in the Azure DevOps service connection. The extension validates this client-side in the service connection UI and also checks it at runtime when waitForCompletion is enabled.

2. Create or Copy the Job Trigger Key

1. Open the Supercheck job you want to run from Azure DevOps
2. Go to that job's CI/CD Settings
3. Create or copy the job trigger key, for example sck_trigger_*

The trigger key is job-specific and is used only to start the run.

3. Add a Service Connection

1. In Azure DevOps, go to Project Settings → Service connections
2. Click New service connection → Supercheck
3. Enter your Supercheck instance URL and CLI token
4. Click Verify and save

4. Use in Pipelines

# Gate a deployment on Supercheck tests passing
steps:
  - task: SupercheckTriggerJob@0
    displayName: 'Run E2E tests'
    inputs:
      supercheckConnection: 'My Supercheck Connection'
      jobId: 'your-job-id-here'
      triggerKey: '$(SUPERCHECK_TRIGGER_KEY)'
      waitForCompletion: true
      timeoutSeconds: 300
      failOnTestFailure: true
      publishJUnitResults: true
      junitResultsFile: '$(Common.TestResultsDirectory)/supercheck/TEST-supercheck.xml'

  - task: PublishTestResults@2
    displayName: 'Publish Supercheck test results'
    condition: always()
    inputs:
      testResultsFormat: 'JUnit'
      testResultsFiles: 'TEST-supercheck.xml'
      searchFolder: '$(Common.TestResultsDirectory)/supercheck'
      mergeTestResults: true
      failTaskOnFailedTests: false
      testRunTitle: 'Supercheck'

Trigger Job Requirements

The Trigger Job task uses two different credentials:

• triggerKey is required and must start with sck_trigger_ or legacy job_
• supercheckConnection is still required for the base URL
• when waitForCompletion: true, the service connection token must be a CLI token beginning with sck_live_

This split mirrors the Supercheck API:

• trigger key to start the run
• CLI token to read run status, run details, and test metadata

JUnit Output And Azure DevOps Tests

When publishJUnitResults is enabled, the Trigger Job task writes a JUnit XML file that can be consumed by PublishTestResults@2.

The generated JUnit includes:

• a synthetic gate testcase for the overall run
• detailed Supercheck testcases when the run API returns testResults
• suite properties such as run ID, job ID, status, duration, and report URL
• enriched testcase metadata from Supercheck test definitions

Tag Conventions For Enriched Test Results

Supercheck tags only allow letters, numbers, underscores, and hyphens. To enrich Azure DevOps test output, use tags like these on your Supercheck tests:

• owner-performance
• service-cabmd15
• area-loadgate
• priority-high

The Trigger Job task maps these into JUnit like this:

• owner-* or owner_* -> Azure DevOps test Owner
• service-*, area-*, and priority-* -> JUnit classname so the test row is easier to group and filter
• test description, priority, and tags -> JUnit system-out, visible in the Azure DevOps test details/logs view

Example metadata to validate the enrichment:

• Description: Load gate for Cab15 release validation
• Tags:
  • owner-performance
  • service-cabmd15
  • area-loadgate
  • priority-high

With that metadata, the imported test result should show:

• Owner: performance
• Test file/class grouping similar to cabmd15/loadgate/high
• Console output containing the description and tags

# Deploy monitoring-as-code config
steps:
  - task: SupercheckDeployConfig@0
    displayName: 'Deploy monitoring config'
    inputs:
      supercheckConnection: 'My Supercheck Connection'
      showDiff: true
      dryRun: false

# Post-deployment health check
steps:
  - task: SupercheckHealthCheck@0
    displayName: 'Verify monitors healthy'
    inputs:
      supercheckConnection: 'My Supercheck Connection'
      checkMonitor: true
      monitorId: 'your-monitor-id'
      waitForMonitorSeconds: 60

Full Pipeline Example

trigger:
  branches:
    include: [main]

stages:
  - stage: Test
    jobs:
      - job: SupercheckE2E
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: SupercheckTriggerJob@0
            displayName: 'Run Supercheck E2E tests'
            inputs:
              supercheckConnection: 'Supercheck Prod'
              jobId: '$(SUPERCHECK_E2E_JOB_ID)'
              triggerKey: '$(SUPERCHECK_E2E_TRIGGER_KEY)'
              waitForCompletion: true
              timeoutSeconds: 600
              failOnTestFailure: true
              publishJUnitResults: true
              junitResultsFile: '$(Common.TestResultsDirectory)/supercheck/TEST-supercheck.xml'
          - task: PublishTestResults@2
            displayName: 'Publish Supercheck test results'
            condition: always()
            inputs:
              testResultsFormat: 'JUnit'
              testResultsFiles: 'TEST-supercheck.xml'
              searchFolder: '$(Common.TestResultsDirectory)/supercheck'
              mergeTestResults: true
              failTaskOnFailedTests: false
              testRunTitle: 'Supercheck E2E'

  - stage: Deploy
    dependsOn: Test
    condition: succeeded()
    jobs:
      - deployment: DeployApp
        environment: production
        strategy:
          runOnce:
            deploy:
              steps:
                - script: echo "Deploy your app here"

            postRouteTraffic:
              steps:
                - task: SupercheckHealthCheck@0
                  displayName: 'Post-deploy health check'
                  inputs:
                    supercheckConnection: 'Supercheck Prod'
                    checkMonitor: true
                    monitorId: '$(SUPERCHECK_PROD_MONITOR_ID)'
                    waitForMonitorSeconds: 30

  - stage: UpdateMonitoring
    dependsOn: Deploy
    condition: succeeded()
    jobs:
      - job: DeployConfig
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: SupercheckDeployConfig@0
            displayName: 'Sync monitoring config'
            inputs:
              supercheckConnection: 'Supercheck Prod'
              showDiff: true

Output Variables

The Trigger Job task sets these pipeline variables when outputRunId is enabled:

Self-Hosted Supercheck

If running a self-hosted Supercheck instance, set the URL in your service connection to your instance's address (e.g., https://supercheck.yourdomain.com).

Links

• Supercheck Documentation
• Supercheck CLI Reference
• Supercheck API Reference

Development

npm install
npm run bump:patch
npm test
npm run build
npm run package

The root scripts now handle TypeScript compilation, trigger-job test compilation, task dependency installation, and VSIX packaging on both Windows and Linux agents.