Introduction

Concurrency Tester is a Visual Studio extension. It executes NUnit tests concurrently thereby simulating multi-user scenario. It is targeted against the integration tests written using NUnit.

User can select the number of concurrent users and run all tests those many times in parallel.

The need

Performance/stress tests are performed after development (coding) is completed. As part of development, a developer writes unit and integration tests, runs them and after successful run, declares “dev. complete”. All these tests (especially integration tests) are executed sequentially.  There are scenarios listed below which are never tested because of thissequential nature of execution.

1. General performance tests under expected load:  As mentioned earlier, the performance testing is done after development is complete. This is just too late in the DevOps pipeline. Developers never get an idea about the performance of their code until it is executed in a multi-user environment. This is too late and at the most inappropriate time because business is eagerly expecting to release the software.
2. Hard to replicate/reproduce stress test scenario:  This is particularly true in scenarios where developers are given a log file produced after carrying out performance test. It’s difficult for a developer to simulate exact scenario by just using log file.
3. Database locking and deadlock issues: Scenario of 2 methods trying to access same database resource is never orchestrated using integration tests.
4. Static class usages:  Since only 1 pass of integration tests is executed (albeit multiple times!), behaviour of static class/methods (extension methods) in a concurrent/multi-user environment is never tested.
5. Database connection pooling issues: Integration test can’t test scenarios where connection pool exceeds allowed limit.

A simple case study

Consider following case study to understand how concurrency tester can help in identifying bugs in multi-user scenario.

Let’s say we have got following fictitious business function.

C#
Edit|Remove
csharp
public class TopSecretBusinessFunction    {        static int commission = 3;         public float TransferMoney(float amount)        {            commission --;             return Convert.ToInt16(amount) / commission;        }    }
public class TopSecretBusinessFunction     {         static int commission = 3;           public float TransferMoney(float amount)         {             commission --;               return Convert.ToInt16(amount) / commission;         }     }
And this function has got following integration test.

[TestFixture]    public class BusinessLayerIntegrationTest    {        [Test]        public void WhenMoneyIsTransferredExpectNoExceptionIsThrown()        {            // Arrange            var topSecretBusinessFunction = new TopSecretBusinessFunction();            // Act            TestDelegate td = () => topSecretBusinessFunction.TransferMoney(5);            // Assert            Assert.DoesNotThrow(td);        }    }

[TestFixture]     public class BusinessLayerIntegrationTest     {         [Test]         public void WhenMoneyIsTransferredExpectNoExceptionIsThrown()          {             // Arrange             var topSecretBusinessFunction = new TopSecretBusinessFunction();              // Act             TestDelegate td = () => topSecretBusinessFunction.TransferMoney(5);              // Assert             Assert.DoesNotThrow(td);         }     }

When executed sequentially (i.e. in a single user environment), no matter however times, this test will always pass.

However, there is a subtle bug in the business function. That will only surface in multi-user environment (Refer tothis link to understand what the bug is).

When you run the concurrency tester on the integration test project, the test case in executed in multi-user environment and the bug is caught as shown below.

Prerequisites

Concurrency tool needs following prerequisites in order to run.

1. Visual Studio 2012/2015: Extension is tested in VS 2012/15 environment.
2. NUnit Integration Tests targeting .NET framework 4.X: Extension needs integration tests written using NUnit.  It will work on NUnit unit tests as well but there is no gain in executing unit tests concurrently. Target framework of tests should be .NET framework 4.X.

How to install

Open Visual Studio 2012/15. Go to “Tools à Extensions and Updates…” menu.

In the new window presented, click “OnlineàVisual Studio Gallery” menu on the left hand side. Type in “concurrency tester” in the search box on the right hand side pane. The search results, “Concurrency Tester” will be shown in the middle pane. Click “Download” to start installation.

Accept the license terms and click “Install” button.

You may need to restart Visual Studio after installation. This is 1 time restart only.

How to run concurrency tester

Once you have installed the “concurrency Tester” extension. Open the target Visual Studio solution. Select the “Integration Test Project” as shown below.

Right click on the “Integration Test Project”, you will see the “Enter Concurrent Users” menu.

Enter the value suitable in your scenario. Typical value should be close to no. of concurrent users allowed by database. This is 2000 in case of SQL Server. Below, I am selecting a 10 user environment.

After entering this value, hit “ENTER”.

The results of running concurrent tests are shown in “Output” window. If this window is not already visible, go to “ViewàOutput” menu of Visual Studio.

Select “Concurrency Tester” from available choices.

Read through the results in pane below.

How it works

Working of Concurrency Tester involves following main steps.

1. Selecting a context project.

This involves selecting integration project and using context (or right click) menu.

2.  Finding the output assembly.

This process involves locating output assembly (DLL) from selected project.

1. Extract NUnIt tests from the assembly.

This step involves using reflection to extract test methods.

1. Create a list of Tasks.

Tasks here are the C# Tasks. Add all test cases to this list. Process is repeated for the number of users as selected for running tests.

1. Execute the tasks in parallel.

This step involves starting all the tasks in tasks list using Task Parallel library.

1. Gather the outcome of each task (including exceptions) and display them.