Visual Studio Web Performance Tests – Dynamically looping over nested data

I was recently doing some load testing of a client’s site, using Visual Studio Load Tests. I was running 1000 simulated users in Azure and measuring the response of the site. This is a great way to see how your site behaves under load. To make the tests meaningful, you want to simulate different users performing typical tasks on the target site. I also wanted to run the same tests against CI (continuous integration) and QA configurations.
When recording load tests, you are basically emulating the load at the HTTP layer – you are sending real requests to the back-end web server, API server, BLOBs server, etc. To get realistic data, you can run the site in say Chrome, and look at the network tab to see which requests are being issued, and then mimic those in your load test.
Below is a sampling of the kinds of requests you might see; fetching js files, images, making XHR API calls, etc.  As you see to the right, some of the responses include JSON data that can be parsed and used in subsequent calls.

Chrome Network Traffic

In the UI, you can click on different Students, and then view grades for different Trimesters:

User Interface

The recorded requests show a pattern:

URLs

 
 
 
Analyzing these and comparing these to the JSON retrieved earlier:

JSON

The URLs are of the form:

 
 
So if code could be written to generate these requests, based on the IDs of the students in the JSON data, and the valid marking period Ids, then I could simulate a parent clicking on each student and reviewing grades for each period. This would be done in two loops: an outer loop for each student, and an inner loop for each period. The number of students depends on which parent has logged in (i.e. the upper bound for the outer loop is varies by test login). The number of grading periods depends on the marking periods of the school in question (some have 3 trimesters, some have 4 quarters, etc.). The inner loop bounds vary by school, which can be different for each student.
Armed with this understanding of the data, we now can create some code that iterates over the data and generates the correct API requests dynamically!

Step 1 – Adding the correct requests to your webtest

Now that we have seen the requests we want in the Chrome Network tab, we want to add these same calls to our webtest, and then change them to be dynamic.
You can manually add each of these requests to your test, you can record them from within Visual Studio, or you can use a handy trick I found:
Start out at your login page in the browser, and bring up Chrome Developer Tools. Make sure Preserve Log is checked. Now, navigate through the UI performing actions. Then, in the Chrome Network Tab, choose “Save as HAR with Content” (HAR is an standardized archive format for HTTP requests):

Save as HAR

Next, open Fiddler, and import the HAR by choosing File, Import Sessions:

Once loaded into Fiddler, you can Export them as a Visual Studio Web Test!

Here is the result in Visual Studio:

Step 2 – parsing the data

Once the calls have been added to a test, you can replace the series of nested calls with two Loops. The basic structure of the test is shown below, where the {{variables}} are context parameters that get substituted for values in run-time. (To get this far, I had to pull off a few different tricks to get authentication to work, including scraping a bearer token out of a response and saving it to a context variable, but that is outside the scope of this blog).
Below can see the outer and inner loops. The outer loop is of the style “While (indexvariable) < x” (where x is set to 1, but you will see how to change this before entering the loop). The inner loop is of style While (somevariable) = x, where we set x to 0, and will use logic to flip the variable from 0 to 1 when we want to exit the loop.

Before the first loop we make an API call to getall
This is the API call that returns that big BLOB of JSON data that was shown before, full of interesting info we need for our loops and URLs. To parse that data, we create a Request Plug-in called GetStudentPlugin, that inherits from WebTestRequestPlugin, and in the PostRequest method, takes the Response BodyString and uses Newtonsoft’s JsonConvert to deserialize the JSON string to an array of Student objects, each with an inner list of MarkingPeriod objects.

The C# declaration of these classes was automatically generated by pasting the response JSON (captured in Chrome Network tab) into json2csharp.com.

JSON2CSharp

Once the plugin has deserialized the JSON, we extract the IDs that we are interested in, and save them to context variables:

We have six different pieces of data we want to capture, and so we define six context variables (just public string properties of the Plugin) to put them in. After this code runs, they will be available for subsequent plugins or context variable evaluations.

Step 3 – setting up the outer loop

For the outer loop, we want to iterate once for each student. Since we don’t know the number of students ahead of time, we set the loop termination value to 1, but then modify it before we get to the loop. This is done by another plugin, the SetLoopLimitPlugin. The Plugin has two properties, IndexVariableName and LimitVariableName:

 
First, we need to find the correct loop to modify. As the test may have several loops, we iterate over them (in wt.ConditionalRuleReferences), and examine the ContextParameterName property of each loop (see below – it is set to “StudentIndex”) – this is the variable the holds the current loop index. By giving each loop a different context parameter name for the loop index, we can find the correct one.

 
Second, we modify the TerminatingValue property of the loop (set to 1 in design time), by setting it to the value of the context variable LimitVariableName, which we have configured to be NumStudents.  As you recall, NumStudents was set in the previous step by the GetStudentsPlugin. As you can see in the screen shot below, the runtime value of the TerminatingValue has changed to 5!

Excellent! Now as we watch the test run, we see it loop once for each of the students, no matter how many there are.

Step 4 – Reading the correct values in each loop iteration

As you recall, in the GetStudentsPlugin, we seeded a few different context variables with various arrays, for instance an array of student ids in the variable StudentIds. Looking at the loop below, you can see an assortment of context variables being used in URLs, for instance {{StudentIds_current}}. In fact, you can see that for each array named “X” we created, it’s current value in the loop is being referenced by a context variable {{X_current}}.

This handy trick is pulled of with the help of yet another plugin, the SetupLoopContextPlugin. You can see it above, and it runs before the first request of each iteration of the loop. It takes two parameters: the name of the loop index variable (StudentIndex), and a list of array names (in this case “StudentIds,GradeLevelIds,SchoolIds,ActiveMarkingperiodIds,FirstAttendanceIds,Year”).

The Plugin first finds the current loop index. Next, it splits the string containing a comma separated list of array names into an actual array of array names (!!!), and iterates over them. For each array name, it gets the corresponding array from the matching context variable, and then places the current value (theArray[loopIndex] into a context variable with the same name as the array, followed by “_current”. So for instance, if loopIndex is 3, then the array StudentIds is found, and it’s value theArray[3] is placed in StudentIds_current.
Once the plugin has been executed at the beginning of each pass though the loop, the context variables XXX_current all have the correct values and can be used in the URLs inside the loop. You can see the runtime values for StudentId below (and also note that in loop 6 it exits before running with status “Condition Not Met”, since we had set the condition While {{StudentIndex}} < 5 (dynamically).

Step 5 – Configuring the inner loop

My first attempt at configuring the inner loop was to use the same approach as the outer loop. I added a plugin that found and modified the TerminatingValue of the inner loop to the correct value for each journey through the outer loop. Sadly, that approach does not work! The test runner appears to evaluate all the nested loop TerminatingValues once and once only, when it reaches the first outer loop. We get once chance to modify that outer loop value, but any changes to the inner loop values are ignored.
For the inner loop, we use a different technique. Here, we configure a loop that is set to run as long as a given context variable has a set value. We can set it to run as long as it equals 0. In each loop, we can set the context variable that is being watched to 1 (or any other non zero value) once we have reached the last element of the inner list. So in effect we use the variable as a semaphore.
Before we can set up the logic, we need to make a call to an API in the system being tested, that returns the list of Courses for the current student and active marking period, something like

https://nhaschools-services-ci.nhatest.com/parent-portal/api/grades/courses/1235385/6

We use the same technique as with parsing students to parse the JSON into an array of Grade objects, and then from these, we extract an array of courseSectionIds. These are stored in a context variable, and then you will notice two more similarly named context variables being set up, each with the prefix LoopName. This pattern is in order to make the code somewhat generic when there are multiple inner loops: the LoopName_ids holds the IDs that you want to iterate over, LoopName_stopflag is set to 0 initially and then to 1 when you want to exit, and the current loop index is set to 0 in LoopName_index, and then incremented on each pass through the loop.

After this API call has been made and the Plugin executed, we have set up the context variables needed to enter the inner loop.

Step 6 – setting the inner loop context

As with the outer loop, on each pass through the inner loop, we need to configure the context variables for the “_current” values. We also need to add the logic to set the semaphore that exits the loop. This can all be done with general purpose code in the SetupInnerLoopPlugin:

The Plugin only needs one parameter, the LoopName, which is set to the same values as in the GetGradesPlugin that prepped the loop: “cs” for course sections. Using this LoopName, the Plugin can retrieve the array of ids in the context variable LoopName_ids (cs_ids), and the current loop index from LoopName_index (cs_index). It then increments that index value and writes it back to the context. If the index value equals the length of the array, we set the stop flag LoopName_stopflag (cs_stopflag) to 1. Otherwise, we set LoopName_current (cs_current) to the correct value in the array of ids (ids[loopindex].
Below, you can see the inner loop run time values. It performs 8 iterations, then exits with cs_stopflag <> 0 (Condition Not Met). In the URLs, you see the last segment varying by course ID withing the loop.

Note, in this case we only needed one array of IDs in the inner loop. If we had needed several different arrays (as we do in the outer loop), we could have used a similar technique, adding an ArrayNames parameter to the plugin, and having it iterate over all arrays and set their current values.

Summing it up

Well, that’s a wrap for a fairly long and detailed blog. You have seen some tips and tricks in how to write dynamic, nested loops that iterate over data returned from the APIs being tested. There are several other tricks needed in the bag, including how to do authorization, bypass hidden fields, and iterate over test credentials to log in with, and then there is the whole topic of running a series of these web tests as a Load Test against your site. We will be happy to show you how!

John Waters

John Waters

As a highly respected and recommended technology leader, entrepreneur, and software architect extraordinaire, John Waters provides guidance for Trailhead teams and projects. His remarkable project requirements gathering and project estimation skills allow him to oversee teams that deliver client projects on time and on budget. John’s background includes over two decades of systems architecting, implementations for a range of high-performance business solutions, and speaking at conferences worldwide.

Related Blog Posts

We hope you’ve found this to be helpful and are walking away with some new, useful insights. If you want to learn more, here are a couple of related articles that others also usually find to be interesting:

Manage Your Windows Applications With Winget

Winget, Microsoft’s native package manager for Windows 10 (version 1709 and later) and Windows 11, offers a streamlined CLI for efficient application management. This blog post introduces Winget’s installation and basic commands for installing, updating, and removing software. It highlights the tool’s ability to manage non-Winget-installed apps and explores curated package lists for batch installations. The post also recommends top Winget packages, noting some may require a paid subscription.

Read More

Our Gear Is Packed and We're Excited to Explore With You

Ready to come with us? 

Together, we can map your company’s software journey and start down the right trails. If you’re set to take the first step, simply fill out our contact form. We’ll be in touch quickly – and you’ll have a partner who is ready to help your company take the next step on its software journey. 

We can’t wait to hear from you! 

Main Contact

This field is for validation purposes and should be left unchanged.

Together, we can map your company’s tech journey and start down the trails. If you’re set to take the first step, simply fill out the form below. We’ll be in touch – and you’ll have a partner who cares about you and your company. 

We can’t wait to hear from you! 

Montage Portal

Montage Furniture Services provides furniture protection plans and claims processing services to a wide selection of furniture retailers and consumers.

Project Background

Montage was looking to build a new web portal for both Retailers and Consumers, which would integrate with Dynamics CRM and other legacy systems. The portal needed to be multi tenant and support branding and configuration for different Retailers. Trailhead architected the new Montage Platform, including the Portal and all of it’s back end integrations, did the UI/UX and then delivered the new system, along with enhancements to DevOps and processes.

Logistics

We’ve logged countless miles exploring the tech world. In doing so, we gained the experience that enables us to deliver your unique software and systems architecture needs. Our team of seasoned tech vets can provide you with:

Custom App and Software Development

We collaborate with you throughout the entire process because your customized tech should fit your needs, not just those of other clients.

Cloud and Mobile Applications

The modern world demands versatile technology, and this is exactly what your mobile and cloud-based apps will give you.

User Experience and Interface (UX/UI) Design

We want your end users to have optimal experiences with tech that is highly intuitive and responsive.

DevOps

This combination of Agile software development and IT operations provides you with high-quality software at reduced cost, time, and risk.

Trailhead stepped into a challenging project – building our new web architecture and redeveloping our portals at the same time the business was migrating from a legacy system to our new CRM solution. They were able to not only significantly improve our web development architecture but our development and deployment processes as well as the functionality and performance of our portals. The feedback from customers has been overwhelmingly positive. Trailhead has proven themselves to be a valuable partner.

– BOB DOERKSEN, Vice President of Technology Services
at Montage Furniture Services

Technologies Used

When you hit the trails, it is essential to bring appropriate gear. The same holds true for your digital technology needs. That’s why Trailhead builds custom solutions on trusted platforms like .NET, Angular, React, and Xamarin.

Expertise

We partner with businesses who need intuitive custom software, responsive mobile applications, and advanced cloud technologies. And our extensive experience in the tech field allows us to help you map out the right path for all your digital technology needs.

  • Project Management
  • Architecture
  • Web App Development
  • Cloud Development
  • DevOps
  • Process Improvements
  • Legacy System Integration
  • UI Design
  • Manual QA
  • Back end/API/Database development

We partner with businesses who need intuitive custom software, responsive mobile applications, and advanced cloud technologies. And our extensive experience in the tech field allows us to help you map out the right path for all your digital technology needs.

Our Gear Is Packed and We're Excited to Explore with You

Ready to come with us? 

Together, we can map your company’s tech journey and start down the trails. If you’re set to take the first step, simply fill out the contact form. We’ll be in touch – and you’ll have a partner who cares about you and your company. 

We can’t wait to hear from you! 

Thank you for reaching out.

You’ll be getting an email from our team shortly. If you need immediate assistance, please call (616) 371-1037.