This extension provides a Build task to integrate in your continuous integration pipeline. This task will upload screenshots to test result reports from sources other than Visual Studio Test (TRX), for instance, from jUnit.
Features
- Upload your screenshot on each test failure
- Customizable screenshot folder
- Get more insight on your test failures
Requirements
- Capture your screenshot on test failure using the following path
automatedTestStorage/automatedTestName
(see json response example)
- Currently only support
.png
images
- The task must be placed after the PublishTestResult task in your pipeline
- If you enabled Fail build on test failure on PublishTestResult task preceding PublishTestResultScreenshot, you will need to also enable the Continue on error to make sure PublishTestResultScreenshot is not skipped.
Getting started
First of all, make sure you have screenshots captured during test failures since this extension only handles their upload to the right test result report. This task was developped for Android UI Testing, but I'm sure it can evolve in the near future to support many other usecases.
Compatibility
Version 0.1.x is compatible with:
- Azure DevOps Service Cloud
- Test type: JUnit (not tested but should also work with other supported types since it uses attributes from the json response)
- Microsoft-hosted agents: macOS 10.14 & 10.13, ubuntu16.04, win2019
NOTE that emulators are currently only available on macOS agents, so for Android UI Testing you will be limited to 10.14 or 10.13, but the task is not dependent on macOS agents.
- Specify your organizationName (usually https://dev.azure.com/`{yourorganization}`)
- Optional: you can change where the task looks for screenshot, defaults to "./app/build/reports/androidTests/connected/screenshots/failures/"
Using YAML:
- task: Genetec.publish-test-result-screenshot.pipeline-extension.PublishTestResultScreenshot@0
displayName: 'Publish test result screenshot'
inputs:
organization: {yourOrganizationName}
screenshotFolder: {yourCustomPath}
Specifications
This task uses azure-devops-node-api which, under the hood, uses the Azure DevOps REST Api.
Task Results
Succeeded
- All the screenshots were successfully uploaded
SucceededWithIssues
(2 self explained possibles messages)
- Some/All screenshots upload failed AND/OR
- Some/All screenshots files were missing
Skipped
- No test failures were found
Error
- Usually related to authentification, connection, networkError etc.
Read the following article to find more about the process I went through before creating this extension. You can also browse the code on the GitHub repo