Parasoft Jtest Extension for Microsoft Azure DevOps

This extension enables you to run code analysis with Parasoft Jtest and review analysis results directly in Azure Pipelines.

Parasoft Jtest is a testing tool that automates software quality practices for Java applications. It uses a comprehensive set of analysis techniques, including pattern-based static analysis, dataflow analysis, metrics, code coverage, and unit testing to help you verify code quality and ensure compliance with industry standards, such as CWE, OWASP, and CERT.

• Request a free trial to receive access to Parasoft Jtest's features and capabilities.
• See the user guide for information about Parasoft Jtest's capabilities and usage.

Please visit the official Parasoft website for more information about Parasoft Jtest and other Parasoft products.

• Quick start
• Example Pipelines
• Configuring Analysis with Jtest
• Task Parameters

Quick start

To analyze your code with Parasoft Jtest and review analysis results in Azure Pipelines, you need to customize your pipeline to include:

• Integration with your build to determine the scope of analysis.
• The task to run Jtest provided by this extension.
• The task to upload the Jtest analysis report in the SARIF format.
• The task to upload the Jtest analysis reports in other formats (XML, HTML, etc.) as pipeline artifacts.

Prerequisites

• This extension requires Parasoft Jtest with a valid Parasoft license.
• The SARIF SAST Scans Tab extension must be installed in your Azure DevOps organization.
• We recommend that you execute the pipeline on a self-hosted runner with Parasoft Jtest installed and configured on the runner.

Installing Required Extensions

1. Sign into the Visual Studio Marketplace and click the Azure DevOps tab.
2. Use the search box to find the Jtest extension.
3. Select the extension and choose Get it free.
4. Select your organization from the drop-down menu and choose Install.
5. Repeat the above steps to install the SARIF SAST Scans Tab extension (if not already installed). This will add a Scans tab to each build result for displaying Jtest analysis results.

Adding the Run Jtest Task to a Pipeline

Add the Run Jtest (RunJtest) task to your pipeline to launch code analysis with Parasoft Jtest.

You need to adjust your pipeline to collect the required input data for Jtest, depending on the build system you are using (Gradle,Maven, or Ant). See Parasoft Jtest User Guide for details.

# Runs code analysis with Jtest.      
- task: RunJtest@1
  inputs:
    workingDir: '$(Build.Repository.LocalPath)'
    dataJson: 'build/jtest/jtest.data.json'

Uploading Analysis Results to Azure Pipelines

By default, the Run Jtest task generates analysis reports in the SARIF, XML, and HTML formats.

When you upload the SARIF report to Azure Pipelines, the results will be presented on the Scans tab. This allows you to review the results of code analysis with Parasoft Jtest directly in Azure Pipleines as part of your project.

To upload the SARIF report, modify your pipeline by adding the PublishBuildArtifacts task. Be sure to use CodeAnalysisLogs as the artifact name.

# Uploads analysis results in the SARIF format, so that they can be accessed in the 'Scans' tab.
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: 'reports/report.sarif'
    ArtifactName: 'CodeAnalysisLogs'
    publishLocation: 'Container'

To upload reports in other formats (.xml, .html), modify your pipeline by adding another PublishBuildArtifacts task. We recommend using JtestReports as the artifact name.

# Uploads all report files (.xml, .html, .sarif) as build artifacts.
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: 'reports'
    ArtifactName: 'JtestReports'
    publishLocation: 'Container'

Example Pipelines

The following examples show simple pipelines for Gradle and Maven projects. The examples assume that Jtest is run on a self-hosted runner (from the self-hosted-jtest pool) and the path to the jtestcli executable is available on PATH.

Run Jtest with Gradle project

# This is an example pipeline to help you get started with the Run Jtest task for a Gradle project.

# Triggers the pipeline on push or pull request events but only for the master (main) branch.
trigger:
  - master
  - main

# Specifies the type of runner that the pipeline will run on.
pool:
  name: self-hosted-jtest


# Configures your Gradle project.  
- task: Gradle@2
  inputs:
    workingDirectory: ''
    gradleWrapperFile: 'gradlew'
    gradleOptions: '-Xmx3072m'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '1.8'
    jdkArchitectureOption: 'x64'
    tasks: 'build jtest'
    options: '-Djtest.skip=true -I path\to\jtest\integration\gradle\init.gradle'

# Runs code analysis with Jtest.   
  - task: RunJtest@1
    inputs:
      workingDir: '$(Build.Repository.LocalPath)'
      dataJson: 'build/jtest/jtest.data.json'

# Uploads analysis results in the SARIF format, so that they can be accessed in the 'Scans' tab.
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: 'reports/report.sarif'
    ArtifactName: 'CodeAnalysisLogs'
    publishLocation: 'Container'

# Uploads all report files (.xml, .html, .sarif) as build artifacts.
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: 'reports'
    ArtifactName: 'JtestReports'
    publishLocation: 'Container'

Run Jtest with Maven project

# This is an example pipeline to help you get started with the Run Jtest task for a Maven project.

# Triggers the pipeline on push or pull request events but only for the master (main) branch.
trigger:
  - master
  - main

# Specifies the type of runner that the pipeline will run on.
pool:
  name: self-hosted-jtest

steps:

# Configures your Maven project.  
- task: Maven@3
  inputs:
    mavenPomFile: 'pom.xml'
    mavenOptions: '-Xmx3072m'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '1.8'
    jdkArchitectureOption: 'x64'
    goals: 'install jtest:jtest'
    options: '-Djtest.skip=true'

# Runs code analysis with Jtest.   
- task: RunJtest@1
  inputs:
    workingDir: '$(Build.Repository.LocalPath)'
    dataJson: 'target/jtest/jtest.data.json'


# Uploads analysis results in the SARIF format, so that they can be accessed in the 'Scans' tab.
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: 'reports/report.sarif'
    ArtifactName: 'CodeAnalysisLogs'
    publishLocation: 'Container'

# Uploads all report files (.xml, .html, .sarif) as build artifacts.
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: 'reports'
    ArtifactName: 'JtestReports'
    publishLocation: 'Container'

Configuring Analysis with Jtest

You can configure analysis with Parasoft Jtest in the following ways:

• By customizing the Run Jtest task directly in your Azure pipeline. See Task Parameters for a complete list of available parameters.
• By configuring options in Parasoft Jtest tool. We recommend creating a jtestcli.properties file that includes all the configuration options and adding the file to Jtest's working directory - typically, the root directory of your repository. This allows Jtest to automatically read all the configuration options from that file. See Parasoft Jtest User Guide for details.

Examples

This section includes practical examples of how the Run Jtest task can be customized directly in the YAML file of your pipeline.

Configuring the Path to the Jtest Installation Directory

If jtestcli executable is not on PATH, you can configure the path to the installation directory of Parasoft Jtest, by configuring the installDir parameter:

- task: RunJtest@1
  inputs:
    installDir: 'opt/parasoft/jtest'

Defining the Scope of Analysis

You can configure the dataJson parameter to provide the path to a JSON file that defines the scope of analysis. Parasoft Jtest User Guide for details.

- task: RunJtest@1
  inputs:
    dataJson: 'target/jtest/jtest.data.json'

Configuring a Jtest Test Configuration

Code analysis with Jtest is performed by using a test configuration - a set of static analysis rules that enforce best coding practices or compliance guidelines. Parasoft Jtest ships with a wide range of built-in test configurations. To specify a test configuration directly in your workflow, add the testConfig parameter to the Run Jtest task and specify the URL of the test configuration you want to use:

- task: RunJtest@1
  inputs:
    testConfig: 'builtin://Demo Configuration'

Limiting the Scope of Analysis

You can limit the scope of analysis to files that are different between the current working branch and the "origin/main" branch by using the additionalParams parameter to specify the following configuration: scope.scontrol.files.filter.mode=branch - the scope of analysis will be limited to files that are different between the current working branch and the main stream of development scope.scontrol.ref.branch=origin/main - the current working branch will be compared with the origin/main branch

Baselining Static Analysis Results

You can configure your Azure pipelines to baseline current analysis results with reference results.

With this configuration, static analysis results generated for the branch to be merged (e.g., feature branch) are compared with the results generated for the reference branch (e.g., integration branch). As a result, only new violations are presented, enabling developers to focus on the issues related to their code changes.

Follow the procedure below to configure your pull request workflow with static analysis results baselining:

1. Configure your reference (integration) branch pipeline and generate the reference report.

• To automatically access the appropriate reference report, your reference pipelines need to follow the naming convention below:

name: PipelineID_$(System.DefinitionId)-RunID_$(Build.BuildId) 

• To produce a reference report, the pipeline needs to include the correct configuration for the static analysis, as described in other sections of this documentation, including publishing the report.

• Run the pipeline to prepare the reference report.

In the resulting analysis scan, the violations from the reference report have the Baseline status: Unchanged and the ones from the feature branches have the Baseline status: New.

Example of reference branch pipeline:

Note: In this example Jtest is running directly from the installation directory without any build system.

trigger:
- main
- master

pool: 
  name: Default

# Set PipelineID and RunID as run name in order to easy refer to this run for getting ref_report
name: PipelineID_$(System.DefinitionId)-RunID_$(Build.BuildId)

steps:

- task: RunJtest@1
  inputs:
    workingDir: '$(System.DefaultWorkingDirectory)'
    installDir: 'opt/parasoft/jtest'
    dataJson: 'demo/demo.data.json'
    reportFormat: 'sarif-azure,xml'

- task: PublishBuildArtifacts@1
  displayName: "Publish analysis logs"
  inputs:
    PathtoPublish: "reports/report.sarif"
    ArtifactName: "CodeAnalysisLogs"

# Uploads all report files (.xml, .html, .sarif) as build artifacts.
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: 'reports'
    ArtifactName: 'JtestReports'
    publishLocation: 'Container'

2. Configure your current (feature) branch pipeline.

• Add a task for downloading the reference report

Add the DownloadPipelineArtifact@2 task into your YAML configuration for the feature branch to download the artifacts containing xml report from the reference run into the pipeline for the feature branch.

• For the pipeline property, set the PipelineID from the name of reference run.
• If the reference report from particular run is needed, for the runVersion property, set the RunID from the name of reference run.

• Configure the static analysis to use downloaded reference report

Note: Use the additionalParams parameter to specify goal.ref.report.file and goal.ref.report.findings.exclude options.

• goal.ref.report.file - reference report file path
• goal.ref.report.findings.exclude=true – the reports from subsequent runs on merging branch will contain only new violations.
• goal.ref.report.findings.exclude=false – the reports from subsequent runs on merging branch will contain only new and unchanged violations.

Example of pipeline for the current (feature) branch:

Note: In this example Jtest is running directly from the installation directory without any build system.

trigger:
- dev1
- dev2

pool: 
  name: Default

steps:

- task: DownloadPipelineArtifact@2
  inputs:
    source: 'specific'
    project: 'example'
    pipeline: 12
    runVersion: latestFromBranch
    runBranch: refs/heads/main
    artifact: 'JtestReports'
    path: 'ref_report_artifact'


- task: RunJtest@1
  name: Jtest
  displayName: "Jtest"
  inputs:
    workingDir: '$(System.DefaultWorkingDirectory)'
    installDir: 'opt/parasoft/jtest'
    dataJson: 'demo/demo.data.json'
    reportFormat: 'sarif-azure'
    additionalParams: |
      goal.ref.report.file=ref_report_artifact/report.xml
      goal.ref.report.findings.exclude=true
    fail: true
  continueOnError: true

- task: PublishBuildArtifacts@1
  displayName: "Publish analysis logs"
  inputs:
    PathtoPublish: "reports/report.sarif"
    ArtifactName: "CodeAnalysisLogs"
    

# Uploads all report files (.xml, .html, .sarif) as build artifacts.
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: 'reports'
    ArtifactName: 'JtestReports'
    publishLocation: 'Container'

Baselining Static Analysis Results in Pull Requests

Follow this procedure below to configure a pull request based on static analysis results.

1. Configure the pull request policy for your target branch.

   1. From the left panel choose Repos -> Branches. Select the target branch for the merge, then click ... -> Branch Policies.
   2. In the Build Validation section click 'Add new build policy'.
   3. In the panel that opens, choose the build pipeline related to this branch.
   4. Set Trigger to 'Automatic (whenever the source branch is updated)'.
   5. Set Policy requirement to 'Required'.

2. Configure the static analysis task fail policy.

If the fail option is set to true, the static analysis task will fail if violations are reported. The pull request will be blocked until the violation is resolved. If the fail option is set to false, the static analysis task will not fail if violations are reported. The pull request will not be blocked.

Note: If goal.ref.report.findings.exclude is set to false and fail is set to true, the pull request will be blocked until all violations (new and unchanged) are resolved.

In order to address the baseline of violations found during pull request in the code being merged, you need to refer to the previously generated reference report obtained using the DownloadPipelineArtifact@2 task. If the build pipeline chosen when configuring the pull request policy is the same as for target branch for merge (i.e. main branch) then it coincides with the pipeline used for producing reference reports. In this scenario the DownloadPipelineArtifact@2 task and applying properties for reference report may be used conditionally only during pull request.

The conditional DownloadPipelineArtifact@2 task and settings related to the reference report used in the common pipeline:

- task: DownloadPipelineArtifact@2
  inputs:
    source: 'specific'
    project: 'example'
    pipeline: 12
    runVersion: latestFromBranch
    runBranch: refs/heads/main
    artifact: 'JtestReports'
    path: 'ref_report_artifact'
  condition: startsWith(variables['Build.SourceBranch'], 'refs/pull/')

- task: RunJtest@1
  inputs:
    workingDir: '$(System.DefaultWorkingDirectory)'
    installDir: 'opt/parasoft/jtest'
    dataJson: 'demo/demo.data.json'
    reportFormat: 'sarif-azure,xml'
    ${{ if startsWith(variables['Build.SourceBranch'], 'refs/pull/') }}:
      additionalParams: |
        goal.ref.report.file=ref_report_artifact/report.xml
        goal.ref.report.findings.exclude=true
      fail: true
  continueOnError: true

Action Parameters

The following inputs are available for this action: