JMeter Test Explorer

JMeter Test Explorer (JTE) is a Visual Studio extension which allows users to discover JMX files from the current solution, execute them directly from Visual Studio and view the results.

JMeter Test Explorer can be used to analyze and measure the performance/load and functional behavior of web applications, web services, and REST APIs.

With this addon, now developers can test the changes without leaving Visual Studio and capture the results for future comparisons.

Prerequisites

The two main prerequisites needed are:

1. Java version 8 or the above
   https://www.java.com/en/download
2. Latest JMeter binaries. Download and unzip the source or binaries into your local machine. https://jmeter.apache.org/download_jmeter.cgi

JMeter Test Explorer Plugin Workflow

JMeter Test Explorer plugin for Visual Studio executes the JMX files created for performing Load and Performance Testing of the Services (mainly Web Applications). JMX files need to be included in the Solution project after which the plugin will scan the entire solution for existing JMX files and provide you option for directly executing the tests. On execution, the results are displayed on the window inside the Visual Studio as well the CSV of the output is generated in the path where the input JMX file is present. The output CSV files can then be used for a variety of purposes. Please find below diagram displaying the overall workflow of the plugin.

Setup

To setup JMeter Test Explorer please follow below steps:

1. Get the VSIX setup from the shared path/ Download plugin from Visual Studio Market Place.

2. Close all the instances of running Visual Studio.

3. Install the plugin setup and start the Visual Studio.

4. The installed plugin can be found under Tools menu.

5. On clicking the plugin, a prompt will open and ask for the path to “jmeter.bat" file which will be subsequently used to run test cases. This jmeter.bat file is present inside the JMeter binaries bin folder which was fetched in the prerequisite step ({Device Folder}\apache-jmeter-{version}}\bin).

6. On selecting the JMeter batch file path actual JMeter Test Explorer toolbox will be displayed.

JMeter Test Explorer Plugin Features

JMeter Test Explorer comes with features which makes it visually interactive to configure and use it. This section will describe the feature which comes with the plugin. Each section in the image below is assigned a numeral. Working/Description of each is detailed after the image.

1. This space contains the list of the JMX present in the solution. Entire solution is scanned for the JMX files and then listed in a hierarchical view in this space.

2. Search Box: User can search for the JMX file with its name. This search feature is an interactive search box, so list filters as and when characters are typed. Currently, search filter work on the typed characters in the search with the Test Name present in JMX file.

3. This Refresh icon refreshes the list by rescanning the entire solution again for the JMX files if any new file is added/deleted/modified.

4. Settings icon opens a context menu which contains the plugin’s configurable menu items.

• Change JMeter Path: This feature can be used to change the batch file path for the JMeter. If the user wants JMeter executables path is changed it needs to be updated for the plugin to work as well. On clicking this menu item, a prompt is opened which contains the current path and version of JMeter being used by the plugin. User can change the path by clicking the Change button and browse the new path.

1. About opens a prompt window which displays all the details for the plugin.

Demo

For demonstrating the actual use of JMeter Test Explorer, we have used Swagger Petstore as sample API. To check out more about Swagger pet store please visit below link: https://petstore.swagger.io/

And if you want to get the source code and run this solution locally, you can get it from below Github repositoryhttps://github.com/swagger-api/swagger-codegen/tree/master/samples/server/petstore/aspnetcore

Visual Studio Solution Overview

To mimic the actual use case behavior copy the Visual Studio Solution(JMXTest) given in the repository above and open it. Below is the image displaying the Solution structure:

API and JMX Files Description

Swagger pet store solution has 3 API controllers :

1. PetAPI

2. StoreAPI

3. UserAPI

We have created JMXs for testing out these controllers. JMX files that we will be using here is created in the hierarchical structure. Below is the expanded view of JMeter tests folder. All the test cases will be tested for the globally available endpoint (https://petstore.swagger.io/v2) in the JMX files,

1. Swagger Petstore.jmx: This is the root JMX for the entire API. It is the main fragment which contains child fragments i.e. ThreadGroups (API endpoints). We are going to run this JMX and which runs the internal JMXs which are referred in this main fragment.

2. User.jmx, Store.jmx & Pet.jmx: These are child fragments to the main fragment. The endpoints are segregated so that they can be re-used and managed independently.

3. CSV Files: The CSV files present inside child fragments folders contains the test data for the individual endpoints present inside the child fragments. This data is then used by the JMeter to perform Load and Performance test.

JMeter Test Explorer

JMeter Test Explorer scans through the solution and creates the same hierarchical structure for the JMXs as present in the folder. In general, the logical hierarchical arrangement for a JMX is in the below format:

For more information regarding the JMX and its components, please visit URL: https://jmeter.apache.org/usermanual/test_plan.html In the plugin, user can see this arrangement in the below structure:

User Actions:

a. Run Test: Runs the test plan as configured at that state.

b. Open in JMeter: Will open the JMeter Application GUI with the JMX file against which it is clicked.

c. Enable/Disable: Sets the enable attribute to true or false for either thread group or sampler.

d. Set User Defined Variables: This can be used to set values to user-defined variables defined in the JMX. These variables are prefixed by ${__P in the file. An example of it is given below

Once this icon is clicked, a form window will open with default value populated in it. User can change these values and Save it in the file directly from the form. And new changes will start reflecting. This form window will look like below:

e. When Run Test action is performed on the test plan,the process in the background will start executing the test plan and will display the result in the window.

f. A new action item is also visible at the rightmost corner with a green tick.

On clicking this Output CSV generated is brought to the front in the application supported for reading CSV files by the system (MS Excel, Notepad++). The output file name is in the format {OriginalFileName}-output-{DateTime}.csv and this file is saved in the same location as the test plan JMX file. An example of it can be viewed in the below image.

You can down load the working VS project from here. https://github.com/emtecinc/JMeter-Test-Runner

g. This output result can be used by the user for further purpose. User can validate if their services are up and running, analyze performance and load test result of this test, publish results, etc.

NOTE: JMeter Test Explorer is supported and tested on Visual Studio version 17(Professional & Enterprise). For earlier versions,the behavior may change while work is under progress to make it fully functional on earlier versions as well.