Extending Azure Test Suite and Test plan to publish results from non Microsoft Technologies e.g. Java/Java Script/Python
Azure DevOps Testing Hub Extension to support Non Dotnet Technologies
Install the extension in your Azure DevOps organization
Create/update a pipeline (build or release)
Go for adding pipeline tasks, and search for "Azure DevOps Testing Extension" and click Add
Follow steps given below for configuring the task for ccreating tests or updating test results
This task has three main configurations:
Select the Operation: To start with we need to select an Operation type either "Create Tests from Feature Files" or "Update Test Results in Test Plan/Suites"
Create Tests From Feature Files: This option is used to create test cases in Azure DevOps, following are the mandatory data to provide to support this operation:
Folder Containing the Feature Files - For dotnet project make sure to provide path for folder containing your project, for others e.g. Java/JS etc provide the parent folder path contaning feature files.
Is it a Dotnet Project? - This option needs to Set to "True" if your project is a dotnet else set it to "false" for other projects.
Update Test Results in Test Plan/Suites: Use this option to update your test execution results to provided test plan/suite based on given test configuration for non dotnet project(s) only. We need to provide following details with this option:
Folder containing the XML Result File - This is the folder containing the test result XML files in JUNit format.
Naming Convention For XML Result File - Naming conventions to filter the XML file from above folder e.g. if your files are generated with TestResults-xxxxxx.xml, then use "TestResults" here as filter value.
Test Plan - Provide ID of Azure Test Plan to publish test results from above XML files
Test suites (comma separated) - Provide a comma separated Ids of test suites under above test plan for search and update test results
Test configuration - Test Configuratio ID for updating the tests
General Settings: This setting is for capturing following details for Create Tests or Update Test Results:
DevOps URL - URL fo your organization
Your Project Name in DevOps - Name of your project where you want the test cases to be create or containing the Test PLan/Suite to update test results
Personal Access Token - It should a PAT with Work Item Management Read/Write access and Test Management Read/Write Access. Make sure to use it as secret variable or key vault
Path for dotnet Exe on the agent running the Job - Default value is set, update if your dotnet is at different location
Name of your repo containing your test code - Default is empty. Note: Adding any value here will be added as prefixed to the created tests.
For publishing the test results in Azure DevOps test plan/suite, please make a note this will only work for non-dotnet project(s) for dotnet project please use Microsoft provided VS Test task. Following are main mandatory steps for projects:
Projects should be generating the test result in JUnit result XML format. More info visit here
Make sure "testcase" element in result XML must match the name of test case title in Azure DevOps to identify the tests.