Properly Tracking Manual Testing in Azure DevOps

Ensuring a software application is thoroughly tested is crucial for most software projects. By conducting comprehensive testing, any potential bugs or defects can be identified and addressed before they have a chance to cause any negative experiences for users.

Today, a typical QA process may consist of the following tasks:

  1. Analyzing requirements
  2. Planning and designing tests
  3. Executing tests and reporting bugs
  4. Running regression tests
  5. Running release tests

There are several tools that can help you run all of the steps above more smoothly, such as Jira, Azure DevOps, Redmine, Asana, etc. In this article, I want to focus on the two most important parts of manual testing–designing your tests and execution of those tests (#2 and #3, bolded above). We’ll look at how to properly set up your Azure DevOps tests to get the most out of your manual testing.

Perhaps in a future blog I can cover how do do the other 3 items above using DevOps.

Plan and Design the Tests

Imagine we have a project in Azure DevOps with some user stories in it. Let’s assume we would like to make sure we cover those user stories with tests before we start testing.

Test Plans

First of all we need a Test Plan. In Azure DevOps, a Test Plan looks more like a list of Test Suits and Test Cases versus the standard use of the term Test Plan with which you may be familiar. Head into the ‘Test Plans’ section and create a New Test Plan.

My preference is to do one Test Plan for each small project or phase of a bigger project (e.g MVP, Phase 2, Phase 3, etc.). However, Azure DevOps makes it quite tricky to work with tests from different test plans.

To get started tracking your tests, let’s look at how to set them up in Test Suites, Test Plans, and Test Cases.

Test Suites

There are 3 types of Test Suites in the Azure DevOps:

  • Static suite
  • Requirements based suite
  • Query based suite

If you want to cover separate User Stories, I would recommend creating a Requirements Based test suite. This type of Test Suite makes a relationship 1-to-1 between a User Story and a Test Suite. Also, it gives you the opportunity to see your tests and Pass/Fail status on the Board for a particular User Story:

A Static suite is the best fits to general purposes, e.g if you want to cover by tests, already existing project, post-factum.

Query Based test suites will help with Sanity/Regression testing. You can sort existing test cases by Priority, Area Path etc. to form the best suite for your purposes.

Test Cases

Let’s add Test Cases to our Test Suites. There are couple of ways to add them to your Suite:

  • Test Plan -> Test Suite -> Define
  • Board -> Add Test

Make sure that each test case has a Priority and Area (functional zones of your app). It will help you to sort test cases for Sanity/Regression suits in the future. Also, it will give you the opportunity to see useful QA Statistics/Charts, regarding your test coverage.

Here are some of the most useful options for Test Cases and QA Statistics in Azure DevOps:

  • Configurations – let you add Browsers, Devices to set them in Test Execution for Cross Browser, Cross Device testing.
  • Parameters – let you set some static values e.g Credentials to Test/Prod environment, that you can use in Test Cases later.

Execute Tests and Report Bugs

Once our Test Suites, Plans, and Cases are ready, you’re ready to execute your tests and track the restules. Do do this, you can simply go to ‘Execute’ tab, set the configuration that you need, and Run your test execution:

While running your tests, make sure that bugs that was found are created inside of Execution pop up. This ensures that these bugs will have correct relationships to make your Charts more accurate in the future. If you don’t log bugs this way, you will need to set the relationships between your QA execution and the bug manually.

As a result, we will have a visualization of our Test Execution for your stakeholders or leadership team on the Board:

And on the default Progress Report:

Also, you can create your won custom Charts for Test Plans/Test Suits and for Bug tracking:

  • Test Plan -> Chart
  • Boards -> Queries -> Charts

Since Charts in Azure DevOps is quite a big topic, I will also review those feature in more detail in a future post.

Summary

With just a little work to make sure you have the proper setup in Azure DevOps projects, you can test and track the quality of your application more easily and accurately, and see how your application’s quality is progressing over time.

Related Blog Posts

We hope you’ve found this to be helpful and are walking away with some new, useful insights. If you want to learn more, here are a couple of related articles that others also usually find to be interesting:

Our Gear Is Packed and We're Excited to Explore With You

Ready to come with us? 

Together, we can map your company’s software journey and start down the right trails. If you’re set to take the first step, simply fill out our contact form. We’ll be in touch quickly – and you’ll have a partner who is ready to help your company take the next step on its software journey. 

We can’t wait to hear from you! 

Main Contact

This field is for validation purposes and should be left unchanged.

Together, we can map your company’s tech journey and start down the trails. If you’re set to take the first step, simply fill out the form below. We’ll be in touch – and you’ll have a partner who cares about you and your company. 

We can’t wait to hear from you! 

Montage Portal

Montage Furniture Services provides furniture protection plans and claims processing services to a wide selection of furniture retailers and consumers.

Project Background

Montage was looking to build a new web portal for both Retailers and Consumers, which would integrate with Dynamics CRM and other legacy systems. The portal needed to be multi tenant and support branding and configuration for different Retailers. Trailhead architected the new Montage Platform, including the Portal and all of it’s back end integrations, did the UI/UX and then delivered the new system, along with enhancements to DevOps and processes.

Logistics

We’ve logged countless miles exploring the tech world. In doing so, we gained the experience that enables us to deliver your unique software and systems architecture needs. Our team of seasoned tech vets can provide you with:

Custom App and Software Development

We collaborate with you throughout the entire process because your customized tech should fit your needs, not just those of other clients.

Cloud and Mobile Applications

The modern world demands versatile technology, and this is exactly what your mobile and cloud-based apps will give you.

User Experience and Interface (UX/UI) Design

We want your end users to have optimal experiences with tech that is highly intuitive and responsive.

DevOps

This combination of Agile software development and IT operations provides you with high-quality software at reduced cost, time, and risk.

Trailhead stepped into a challenging project – building our new web architecture and redeveloping our portals at the same time the business was migrating from a legacy system to our new CRM solution. They were able to not only significantly improve our web development architecture but our development and deployment processes as well as the functionality and performance of our portals. The feedback from customers has been overwhelmingly positive. Trailhead has proven themselves to be a valuable partner.

– BOB DOERKSEN, Vice President of Technology Services
at Montage Furniture Services

Technologies Used

When you hit the trails, it is essential to bring appropriate gear. The same holds true for your digital technology needs. That’s why Trailhead builds custom solutions on trusted platforms like .NET, Angular, React, and Xamarin.

Expertise

We partner with businesses who need intuitive custom software, responsive mobile applications, and advanced cloud technologies. And our extensive experience in the tech field allows us to help you map out the right path for all your digital technology needs.

  • Project Management
  • Architecture
  • Web App Development
  • Cloud Development
  • DevOps
  • Process Improvements
  • Legacy System Integration
  • UI Design
  • Manual QA
  • Back end/API/Database development

We partner with businesses who need intuitive custom software, responsive mobile applications, and advanced cloud technologies. And our extensive experience in the tech field allows us to help you map out the right path for all your digital technology needs.

Our Gear Is Packed and We're Excited to Explore with You

Ready to come with us? 

Together, we can map your company’s tech journey and start down the trails. If you’re set to take the first step, simply fill out the contact form. We’ll be in touch – and you’ll have a partner who cares about you and your company. 

We can’t wait to hear from you! 

Thank you for reaching out.

You’ll be getting an email from our team shortly. If you need immediate assistance, please call (616) 371-1037.