(616) 371-1037

[email protected]

Visual Studio Web Performance Tests – Dynamically looping over nested data

January 30, 2018 - John Waters

No Comments

I was recently doing some load testing of a client’s site, using Visual Studio Load Tests. I was running 1000 simulated users in Azure and measuring the response of the site. This is a great way to see how your site behaves under load. To make the tests meaningful, you want to simulate different users performing typical tasks on the target site. I also wanted to run the same tests against CI (continuous integration) and QA configurations.

When recording load tests, you are basically emulating the load at the HTTP layer – you are sending real requests to the back-end web server, API server, BLOBs server, etc. To get realistic data, you can run the site in say Chrome, and look at the network tab to see which requests are being issued, and then mimic those in your load test.

Below is a sampling of the kinds of requests you might see; fetching js files, images, making XHR API calls, etc.  As you see to the right, some of the responses include JSON data that can be parsed and used in subsequent calls.

Chrome Network Traffic

In the UI, you can click on different Students, and then view grades for different Trimesters:

User Interface

The recorded requests show a pattern:

URLs

 

 

 

Analyzing these and comparing these to the JSON retrieved earlier:

JSON

The URLs are of the form:

 

 

So if code could be written to generate these requests, based on the IDs of the students in the JSON data, and the valid marking period Ids, then I could simulate a parent clicking on each student and reviewing grades for each period. This would be done in two loops: an outer loop for each student, and an inner loop for each period. The number of students depends on which parent has logged in (i.e. the upper bound for the outer loop is varies by test login). The number of grading periods depends on the marking periods of the school in question (some have 3 trimesters, some have 4 quarters, etc.). The inner loop bounds vary by school, which can be different for each student.

Armed with this understanding of the data, we now can create some code that iterates over the data and generates the correct API requests dynamically!

Step 1 – Adding the correct requests to your webtest

Now that we have seen the requests we want in the Chrome Network tab, we want to add these same calls to our webtest, and then change them to be dynamic.

You can manually add each of these requests to your test, you can record them from within Visual Studio, or you can use a handy trick I found:

Start out at your login page in the browser, and bring up Chrome Developer Tools. Make sure Preserve Log is checked. Now, navigate through the UI performing actions. Then, in the Chrome Network Tab, choose “Save as HAR with Content” (HAR is an standardized archive format for HTTP requests):

Save as HAR

Next, open Fiddler, and import the HAR by choosing File, Import Sessions:

Once loaded into Fiddler, you can Export them as a Visual Studio Web Test!

Here is the result in Visual Studio:

Step 2 – parsing the data

Once the calls have been added to a test, you can replace the series of nested calls with two Loops. The basic structure of the test is shown below, where the {{variables}} are context parameters that get substituted for values in run-time. (To get this far, I had to pull off a few different tricks to get authentication to work, including scraping a bearer token out of a response and saving it to a context variable, but that is outside the scope of this blog).

Below can see the outer and inner loops. The outer loop is of the style “While (indexvariable) < x” (where x is set to 1, but you will see how to change this before entering the loop). The inner loop is of style While (somevariable) = x, where we set x to 0, and will use logic to flip the variable from 0 to 1 when we want to exit the loop.

Before the first loop we make an API call to getall

This is the API call that returns that big BLOB of JSON data that was shown before, full of interesting info we need for our loops and URLs. To parse that data, we create a Request Plug-in called GetStudentPlugin, that inherits from WebTestRequestPlugin, and in the PostRequest method, takes the Response BodyString and uses Newtonsoft’s JsonConvert to deserialize the JSON string to an array of Student objects, each with an inner list of MarkingPeriod objects.

The C# declaration of these classes was automatically generated by pasting the response JSON (captured in Chrome Network tab) into json2csharp.com.

JSON2CSharp

Once the plugin has deserialized the JSON, we extract the IDs that we are interested in, and save them to context variables:

We have six different pieces of data we want to capture, and so we define six context variables (just public string properties of the Plugin) to put them in. After this code runs, they will be available for subsequent plugins or context variable evaluations.

Step 3 – setting up the outer loop

For the outer loop, we want to iterate once for each student. Since we don’t know the number of students ahead of time, we set the loop termination value to 1, but then modify it before we get to the loop. This is done by another plugin, the SetLoopLimitPlugin. The Plugin has two properties, IndexVariableName and LimitVariableName:

 

First, we need to find the correct loop to modify. As the test may have several loops, we iterate over them (in wt.ConditionalRuleReferences), and examine the ContextParameterName property of each loop (see below – it is set to “StudentIndex”) – this is the variable the holds the current loop index. By giving each loop a different context parameter name for the loop index, we can find the correct one.

 

Second, we modify the TerminatingValue property of the loop (set to 1 in design time), by setting it to the value of the context variable LimitVariableName, which we have configured to be NumStudents.  As you recall, NumStudents was set in the previous step by the GetStudentsPlugin. As you can see in the screen shot below, the runtime value of the TerminatingValue has changed to 5!

Excellent! Now as we watch the test run, we see it loop once for each of the students, no matter how many there are.

Step 4 – Reading the correct values in each loop iteration

As you recall, in the GetStudentsPlugin, we seeded a few different context variables with various arrays, for instance an array of student ids in the variable StudentIds. Looking at the loop below, you can see an assortment of context variables being used in URLs, for instance {{StudentIds_current}}. In fact, you can see that for each array named “X” we created, it’s current value in the loop is being referenced by a context variable {{X_current}}.

This handy trick is pulled of with the help of yet another plugin, the SetupLoopContextPlugin. You can see it above, and it runs before the first request of each iteration of the loop. It takes two parameters: the name of the loop index variable (StudentIndex), and a list of array names (in this case “StudentIds,GradeLevelIds,SchoolIds,ActiveMarkingperiodIds,FirstAttendanceIds,Year”).

The Plugin first finds the current loop index. Next, it splits the string containing a comma separated list of array names into an actual array of array names (!!!), and iterates over them. For each array name, it gets the corresponding array from the matching context variable, and then places the current value (theArray[loopIndex] into a context variable with the same name as the array, followed by “_current”. So for instance, if loopIndex is 3, then the array StudentIds is found, and it’s value theArray[3] is placed in StudentIds_current.

Once the plugin has been executed at the beginning of each pass though the loop, the context variables XXX_current all have the correct values and can be used in the URLs inside the loop. You can see the runtime values for StudentId below (and also note that in loop 6 it exits before running with status “Condition Not Met”, since we had set the condition While {{StudentIndex}} < 5 (dynamically).

Step 5 – Configuring the inner loop

My first attempt at configuring the inner loop was to use the same approach as the outer loop. I added a plugin that found and modified the TerminatingValue of the inner loop to the correct value for each journey through the outer loop. Sadly, that approach does not work! The test runner appears to evaluate all the nested loop TerminatingValues once and once only, when it reaches the first outer loop. We get once chance to modify that outer loop value, but any changes to the inner loop values are ignored.

For the inner loop, we use a different technique. Here, we configure a loop that is set to run as long as a given context variable has a set value. We can set it to run as long as it equals 0. In each loop, we can set the context variable that is being watched to 1 (or any other non zero value) once we have reached the last element of the inner list. So in effect we use the variable as a semaphore.

Before we can set up the logic, we need to make a call to an API in the system being tested, that returns the list of Courses for the current student and active marking period, something like

We use the same technique as with parsing students to parse the JSON into an array of Grade objects, and then from these, we extract an array of courseSectionIds. These are stored in a context variable, and then you will notice two more similarly named context variables being set up, each with the prefix LoopName. This pattern is in order to make the code somewhat generic when there are multiple inner loops: the LoopName_ids holds the IDs that you want to iterate over, LoopName_stopflag is set to 0 initially and then to 1 when you want to exit, and the current loop index is set to 0 in LoopName_index, and then incremented on each pass through the loop.

After this API call has been made and the Plugin executed, we have set up the context variables needed to enter the inner loop.

Step 6 – setting the inner loop context

As with the outer loop, on each pass through the inner loop, we need to configure the context variables for the “_current” values. We also need to add the logic to set the semaphore that exits the loop. This can all be done with general purpose code in the SetupInnerLoopPlugin:

The Plugin only needs one parameter, the LoopName, which is set to the same values as in the GetGradesPlugin that prepped the loop: “cs” for course sections. Using this LoopName, the Plugin can retrieve the array of ids in the context variable LoopName_ids (cs_ids), and the current loop index from LoopName_index (cs_index). It then increments that index value and writes it back to the context. If the index value equals the length of the array, we set the stop flag LoopName_stopflag (cs_stopflag) to 1. Otherwise, we set LoopName_current (cs_current) to the correct value in the array of ids (ids[loopindex].

Below, you can see the inner loop run time values. It performs 8 iterations, then exits with cs_stopflag <> 0 (Condition Not Met). In the URLs, you see the last segment varying by course ID withing the loop.

Note, in this case we only needed one array of IDs in the inner loop. If we had needed several different arrays (as we do in the outer loop), we could have used a similar technique, adding an ArrayNames parameter to the plugin, and having it iterate over all arrays and set their current values.

Summing it up

Well, that’s a wrap for a fairly long and detailed blog. You have seen some tips and tricks in how to write dynamic, nested loops that iterate over data returned from the APIs being tested. There are several other tricks needed in the bag, including how to do authorization, bypass hidden fields, and iterate over test credentials to log in with, and then there is the whole topic of running a series of these web tests as a Load Test against your site. We will be happy to show you how!

John Waters

Leave a comment

Your email address will not be published. Required fields are marked *