Azure DevOps Testing Hub Extension to support Non Dotnet Technologies

Overview

This Build/Release Task is mainly for supporting creating Azure DevOps tests from your BDD/Feature Files irrespective of technology (dotnet or non-dotnet), Automated Test Automation association (for dotnet projects) and lastly support for updating test results to specified Test Plan/Test Suites for dotnet and non-dotnet technologies e.g. JavaScript/Python/Java etc. It provided similar test result management as DevOps provides for dotnet automated tests.

Usage

• Install the extension in your Azure DevOps organization
• Create/update a pipeline (build or release)
• Go for adding pipeline tasks, and search for "Azure DevOps Testing Extension" and click Add
• Follow steps given below for configuring the task for ccreating tests or updating test results

Configuration Options

This task has three main configurations: Select the Operation: To start with we need to select an Operation type either "Create Tests from Feature Files" or "Update Test Results in Test Plan/Suites"

• Create Tests From Feature Files: This option is used to create test cases in Azure DevOps, following are the mandatory data to provide to support this operation:

  • Folder Containing the Feature Files - For dotnet project make sure to provide path for folder containing your project, for others e.g. Java/JS etc provide the parent folder path contaning feature files.
  • Is it a Dotnet Project? - This option needs to Set to "True" if your project is a dotnet else set it to "false" for other projects.

• Update Test Results in Test Plan/Suites: Use this option to update your test execution results to provided test plan/suite based on given test configuration for non dotnet project(s) only. We need to provide following details with this option:

  • Folder containing the XML Result File - This is the folder containing the test result XML files in JUNit format.
  • Naming Convention For XML Result File - Naming conventions to filter the XML file from above folder e.g. if your files are generated with TestResults-xxxxxx.xml, then use "TestResults" here as filter value.
  • Test Plan - Provide ID of Azure Test Plan to publish test results from above XML files
  • Test suites (comma separated) - Provide a comma separated Ids of test suites under above test plan for search and update test results
  • Test configuration - Test Configuratio ID for updating the tests

• General Settings: This setting is for capturing following details for Create Tests or Update Test Results:

  • DevOps URL - URL fo your organization
  • Your Project Name in DevOps - Name of your project where you want the test cases to be create or containing the Test PLan/Suite to update test results
  • Personal Access Token - It should a PAT with Work Item Management Read/Write access and Test Management Read/Write Access. Make sure to use it as secret variable or key vault
  • Path for dotnet Exe on the agent running the Job - Default value is set, update if your dotnet is at different location
  • Name of your repo containing your test code - Default is empty. Note: Adding any value here will be added as prefixed to the created tests.

Important Note

For publishing the test results in Azure DevOps test plan/suite, please make a note this will only work for non-dotnet project(s) for dotnet project please use Microsoft provided VS Test task. Following are main mandatory steps for projects:

• Projects should be generating the test result in JUnit result XML format. More info visit here
• Make sure "testcase" element in result XML must match the name of test case title in Azure DevOps to identify the tests.