Stark Accessibility Audit Task

Table of Contents

• Connecting with Stark Projects
• Technical Details
• Troubleshooting

Connecting with Stark Projects

Configuring your Stark project

Begin by creating a project in Stark and choosing Import Azure DevOps repository.

Copy the provided token and keep it in a safe place for now. You can continue to the next section now. Or optionally, if you want to be able to initiate your pipeline from within Stark, continue with this section.

Create a Personal Access Token. To do this, click on the settings icon next to your user avatar in Azure DevOps, then click on "Personal access tokens". Create a new token with the "Build (Read & execute)" permission. Be sure to copy the token before leaving the dialog, as you will need it.

Create a pipeline if you haven't already. More details are in the next section.

In your Stark project, populate the Pipeline URL and Personal Access Token fields.

Setting up your pipeline

To use Stark Projects to scan your Azure DevOps repositories for accessibility issues, you’ll need to set up custom pipelines in those repositories.

Begin by creating an Azure DevOps pipeline file in the root of your repository: azure-pipelines.yml. Alternatively, you can choose your repository in the New pipeline workflow to create a template YAML file. Choose the Starter pipeline template on the configuration step for a basic file.

Configure your workflow to start your webserver in the background (see example using Node.js below).

Add a step that uses the Stark Accessibility Audit Task (see example in the below YAML file). Paste the Stark token you copied earlier into the token input of the StarkAccessibilityAudit task. If you need to, you can edit your Azure DevOps asset in your Stark project to see your token again.

You can use the following template for your workflow, updating the appropriate values as indicated:

# Stark Accessibility Audit
# https://aka.ms/yaml

trigger:
  # See https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/trigger?view=azure-pipelines for more information.
  - none

pool:
  vmImage: ubuntu-latest

steps:
  # REQUIRED: include this script to set up Playwright, which the Stark Accessibility Audit uses to load your URL(s).
  # For security, consider pinnning Playwright to a specific version. Example: npm install -g playwright@1.58
  - script: npx playwright install chromium --no-shell
    displayName: 'Install Playwright'

  # This is where you execute the commands needed to build and run your server locally in the background.
  # NOTE: the '&' at the end of the 'npm start &' command is needed to ensure the pipeline can continue to the next step.
  - script: |
      npm install
      npm start &
    displayName: 'Build and run server'

  # This is where you configure your Stark Accessibility Audit.
  - task: StarkAccessibilityAudit@2
    inputs:
      # [Required] The Stark token provided when you created your project on the Stark web app.
      # Example: 'stark_705199c1bfc34311b616c92b0748f4d0'
      token: ''

      # [Required] A list of URLs to be scanned, with each URL on its own line.
      # This value follows YAML conventions for multiline strings.
      # NOTE: This is the likely the only value you have to change!
      urls: |
        http://localhost:3000

      # [Optional] The number of milliseconds to wait before your app is ready.
      # Defaults to 5000
      waitTime: 5000

      # [Optional] Minimum score for the action to pass. A number from 0 to 100.
      # Defaults to 0
      minScore: 0

      # [Optional] If a URL scan failed, scans the next one without failing the task.
      # Defaults to false
      skipErrors: false

      # [Optional] Adds a delay in milliseconds before running the scan. This is different from the timeout and delay in the sense that this delay occurs after the page is navigated to.
      # Defaults to 100 ms
      scanDelay: 100

      # [Optional] Specifies browser viewport size using [width]x[height].
      # Defaults to 800x600
      viewport: '800x600'

The Stark task offers convenient arguments for scanning your repository. For most builds, the only arguments you’ll need to configure are token and urls.

As with any Azure DevOps pipeline, you have all the power — all the task needs are URLs to scan. You’re free to configure your pipeline however else you want.

Once you’ve configured and committed your pipeline to the default branch of your repository, you can now trigger the pipeline to begin a scan. Results will be sent back to your Stark project.

Technical Details

Internally, the Stark Azure DevOps task uses Playwright to control the Chrome installation on the ubuntu-latest Azure DevOps image. It will render the provided URLs in order to run the Stark Accessibility Audit for each.

Troubleshooting

If you get an error in Stark when starting the scan and you know that your token is valid: it could be that your user's group doesn't have permissions to allow your pipeline to be executed via external API. To fix this:

1. Navigate to your pipeline.
2. Click on the 3-dot icon on the top right (next to the Edit and Run pipeline buttons).
3. Click on Manage security.
4. Choose your group or user, and ensure that the Edit queue build configuration privilege is set to Allow or Allow (inherited).
5. You may need to recreate your Personal Access Token after this.