Known Issues | Support | YAML | Policies | Task Parameters | Task Variables | Common Usage Scenarios | Policy Results | FAQ
Build Quality Checks
The Build Quality Checks task allows you to add quality gates to your build process.
You can find the changes notes for this task here.
- We are phasing out Team Foundation Server 2015. If you're still using Team Foundation Server 2015, please stay on version 3.x of the
extension which can be downloaded here. For newer versions of Team Foundation Server or Azure DevOps Services please use version 4.x and higher.
- If you run your build agent behind a proxy server, the task will fail and you will see
connect ETIMEDOUT somewhere in the
task log, even if you configured the agent correctly as described here.
To fix this issue, create two environment variables called
HTTPS_PROXY (either in system scope or the user scope of
your agent service account) and set their value to your proxy server address including the port (e.g., http://myproxy:8080).
Note: Build tasks currently do not support proxy authentication.
If you need help with the extension or run into issues, please contact us at firstname.lastname@example.org or create an issue here.
Adding the Task to a Build Definition
The Build Quality Checks (in task category Build) task needs to be placed after the tasks it should inspect. In a Visual Studio
build definition, e.g., an appropriate place would be after build, test, and symbol indexing/publishing. This ensures that, even if
the policy breaks the build, you still get test results as well as the compile output and symbols.
Note: When you want to use the Code Coverage Policy you need to make sure that you publish code coverage values calculated by your test tool first. The Policy itself does not calculate the code coverage. Before you add the task and activate the policy, please make sure that you can already see code coverage values in your build summary. If you are using a test tool other then MSTest (i.e., the Visual Studio Test task), please use the Publish Code Coverage Results task to publish your coverage data before you add the policy.
Adding the Task to a YAML Build Definition
To add the Build Quality Checks task to a YAML build definition, use the fully qualified task name and major version like this
- task: mspremier.BuildQualityChecks.QualityChecks-task.BuildQualityChecks@5 and set a display name using the
displayName property. Then add all task inputs as described under Task Parameters or in the Policies section.
A sample YAML representation of the task is shown below.
- task: mspremier.BuildQualityChecks.QualityChecks-task.BuildQualityChecks@5
displayName: 'Check build quality'
runTitle: 'Run Title'
The Build Quality Checks task currently supports two policies (click the link for details):
- Warnings Policy - Allows
you to fail builds based on the number of build warnings.
- Code Coverage Policy -
Allows you to fail builds based on the code coverage value of your tests.
If you choose
Previous Value for the Fail Build On option for one of the policies, the policy value (e.g., number of warnings)
is, by default, compared to the corresponding value from the last build that ran for the current build definition. If the build
definition targets a Git repository hosted in Azure DevOps Services or Team Foundation Server, the policy will look for the
last build that ran against the same branch as the current build. This behavior can be customized with the following parameters:
Include Partially Succeeded Builds: Uncheck this option if policy values should only be compared to successful baseline builds.
In most cases, including partially succeeded builds is the best option, so you can use the default setting.
YAML: includePartiallySucceeded - Default value is true. Set to false to deactivate the option.
Definition Filter If you have lots of build definitions, it can get hard to find the right one in the Build Definition
drop-down list. You can use the Definition Filter to limit the number of definitions shown in the list. Either enter a specific definition name or
use the asterisk (*) wildcard to search for definitions starting with (e.g., Def*), ending with (e.g., *def), or containing (e.g., *def*) a specific
value. If you want to list all build definitions in your project, use the default value "*".
YAML: baseDefinitionFilter - This parameter is ignored in YAML definitions.
Build Definition: Select the build definition that should be used to search for the baseline build. If you do not set a value,
the last build of the current build definition will be used when comparing policy values. If the drop-down list is empty, please
click the refresh icon to reload the list of available build definitions. The drop-down list shows a maximum of 1000 build definitions. If your definition
is not visible, please enter the build definition ID manually. See
TFVC Topic Branches
for examples for when using a different build definition might be useful.
YAML: baseDefinitionId - Default is empty. Provide a build definition ID.
Note: To always compare policy values to builds from the current build definition that target a specific branch, you need to
choose the current build definition here and then select the appropriate values for Repository and Branch (Git).
Repository: Select the repository that is used to search for baseline branches. The drop-down list is populated after you chose
the Build Definition and will always contain the repository that the selected build definition is connected to. If the drop-down
list is empty after selecting the Build Definition, please click the refresh icon to reload the repository information.
YAML: baseRepoId - This parameter is ignored in YAML definitions.
Note: When you change the Build Definition after selecting a repository, you might see a GUID value in the repository parameter.
This is a refresh limitation of the build UI. Please select the repository again to correct this.
Branch (Git): Select the branch that should be used to search for the baseline build. If you do note set a value, the last build
targeting the currently built branch will be used when comparing policy values. If the drop-down list is empty after selecting the
Repository, please click the refresh icon to reload the list of available branches. Branches are shown with their Git ref name, e.g.
refs/heads/master or refs/heads/myTopicBranch. See
Pull Request Policy Builds
for examples for when using a different branch might be useful.
YAML: baseBranchRef - Default is empty. Provide a Git branch ref (e.g., refs/heads/master).
Note: When you change the Build Definition and Repository after selecting a branch, you need to select the branch again.
Otherwise, a wrong branch name may be saved to the task configuration. This is a refresh limitation of the build UI.
Note: Whenever you choose a different baseline branch, make sure that the retention policy
is configured to keep at least one successful build for your baseline branch!
If you are using multiple Build Quality Checks tasks within the same build, you may use the Run Title parameter to specify a title that
is associated with a specific instance of the task. This will help distinguishing between the task results in the summary section. The run
title is added to the subsection header in the summary in the format <Build Job Name> - <Run Title>.
YAML: runTitle - Default is empty. Provide a run title.
Disable NodeJS certificate check: Check this option if your Team Foundation Server is using a self-signed or corporate SSL certificate
and your build agent version is lower than 2.117.0. The option disables the certificate chain validation of NodeJS. Please read here for details.
YAML: disableCertCheck - Default is false. Set to true to activate the option.
In addition to parameters visible in the task UI there are a few variables you can set to affect the tasks behavior:
BQC.ForceNewBaseline: You can set this variable (usually when queueing a new build) to true to force the policies to ignore an existing baseline build. This
essentially creates a new baseline for future builds. This is especially helpful if there was a larger refactoring or redesign that resulted in a big change in code coverage and you need to establish a new base value. The default value is false.
BQC.LogRawData: You can set this variable to true to enable logging of raw JSON data read by the task. In most cases you should not use this option as it does
increase the log size of the task and may show sensitive data in the log file. However, this data is very helpful in support scenarios. Note that this variable only takes effect if the variable System.Debug is also set to true. The default value is false.
Common Usage Scenarios
The Build Quality Checks task creates its own summary section in the build summary view. This section displays all success,
warning, and error messages for all activated policies. If you run a multi-configuration build, a subsection for each build
job is created so you can see exactly which configuration might have quality issues.
Note: Please see the Limitations and Special Cases
section of the Code Coverage Policy for possible issues with multi-configuration builds and code coverage.
We have put together a list of frequently asked questions and answers in our FAQ document.
If you feel we need to add a specific question to the list, feel free to send it to our support address.
Checklist board icon | Icon designed by Vexels.com