If you work in or with QA, you’ve likely heard the phrase “regression testing.” For most apps and websites, regression testing is done before every new release. In an Agile QA process, it’s a regular part of the QA testing vocabulary. But what is regression testing, and how do you do it?

The Definition of Regression TestingPerson regression testing mobile app

To put it simply, regression testing means testing virtually everything in the software. Not just a new feature, and not just each general section – but every button, flow, and interaction in the app or website.

The official definition of the word “regression” is:

a return to a former or less developed state

In software, a “regression” is a bug in an area of the app or website that was previously working.

Regression testing is the most thorough type of software testing, and can be done with manual and automated testing.

When to Do Regression Testing

Regression testing should be performed anytime there’s a new feature or update. This is to ensure that the changes in the code don’t cause issues with existing features.

website, mobile app, and tablet screensFor example, let’s say that an app has a video player. For an upcoming release, the team adds a feature that allows users to download videos. The developers updates code in the video player section, enabling a “download” option on all videos.

Sounds pretty straightforward, right? However, this new code could end up unintentionally affecting other areas of the video player (or app in general). For example, it’s possible that the new “download” functionality could prevent the video from playing in-app.

Software is dependent on programming language(s) telling it what to do with each interaction. Many lines of code also reference other parts of the overall code. As a result, one tiny change can have huge consequences.

Regression Testing vs. Smoke Testing

Smoke testing is another common type of software testing. Unlike regression testing, smoke testing doesn’t require thoroughly testing every section. Instead, smoke testing mainly focuses on popular flows within the app/website.

Smoke testing can be done anytime. But there are a few main scenarios in which it’s typically done:

1. When a new build (version of an app or website) is ready for testing. If QA can do a quick smoke test first, they can make sure the build is ready for a deeper level of testing. If they end up finding that basic functionality like login or subscribing doesn’t work, they can send it back to development before wasting time.Woman placing a checkmark on a smoke testing results board

2. When regression testing is complete, right before or after launch. Ideally, regression testing should have turned up any potential bugs. But no one is perfect – whether that’s a manual tester or automated script. As a result, it’s generally considered a good idea to do a final smoke test right before launching the new version. (This is also often done around user acceptance testing.)

Once it’s officially live, it’s good to do another smoke test. This can help ensure that the deployment didn’t cause any issues in the production environment.

Learn more in our guide to smoke testing.

Regression Testing With Test Cases

Regression testing is easiest when test cases are involved. Test cases explain the behavior that’s expected from any given interaction with the app or website. When QA has a detailed list of steps to test, it ensures that important areas aren’t getting left out.

To find out more, see our guide to test cases.

Smoke Testing vs. Regression Testing: Test Case Comparisons

Test cases for regression testing are different than those for smoke testing.

Man holding a checklist of test casesFor example, these app test cases are an example of smoke testing login functionality:

  • Tapping “Login/Sign Up” in the left nav menu should bring up the login screen.
  • If user enters valid credentials and taps “Login,” they should be logged in and taken to the dashboard.
  • If user enters invalid credentials and taps “Login,” an error message should appear prompting the user to enter accurate email and/or password.

But with regression testing, login test cases might also include:

  • If user is already logged in to the previous version, when they install the new build version they should still be logged in.
  • When logged in, tapping “Logout” in left nav menu should log the user out.
  • If user enters an email address that does not exist (with any password) and selects Login, relevant error message should come up.
  • If user enters a valid existing account email address with an incorrect password and selects Login, error message should come up.
  • When user is logged in, and closes app or restarts device then re-opens, they should still be logged in.
  • Email address field should not be case sensitive.
  • Password field should be case sensitive.
  • Login button should be grey until user enters credentials (whether valid or not), at which point it should turn green.

Along with even more scenarios, in many cases.

Regression Testing Without Test Cases

In a fast-paced Agile team, chances are you won’t always have time to write test cases. However, that doesn’t mean there’s no regression testing.

Even under tight deadlines, QA can usually spend a few minutes making a list of sections/features. They can then use this list as a reference throughout regression testing. The list might not cover every single possible behavior. But at least it will guarantee that no section is completely skipped.

Regression Testing TimelineThree people looking at a regression testing list next to a clock

Sometimes QA is expected to finish testing in a time frame that doesn’t allow for full regression. In fact, this isn’t uncommon at all. On most Agile teams, Sprints rarely allow enough time to fully test the app from top to bottom.

So what do you do in these situations? These steps can help:

1. Talk to your manager or client. You’ll want to (respectfully) make it clear that you won’t be able to fully test every possible interaction in the given time. But you can also convey that you’ll do the best you can, and be as thorough as the time allows. It’s also a good idea to be prepared to give an estimate, in case they ask how much time you would need to fully test it.

2. Do a combination of regression and smoke testing. That is, more thorough than regular smoke testing, but without checking every possible edge case.

What if You’re New to the Team?

Regression testing is easiest when you already know the software like the back of your hand. But sometimes you’ll be new to a team or project. So practically speaking, that level of familiarity isn’t always possible.

If you’re new to the team and tasked with doing regression testing, don’t panic! You have options:

Two people discussing regression testing with a mobile app

1. Create a suite of test cases, if they don’t already exist. Even if you don’t have acceptance criteria, you can still write test cases without requirements. That way, you’ll have a guide to each action you need to test.

2. If there are other QA testers on the team, ask for a walk-through to see how they tackle it. If there are no other testers, you can often ask a developer for a more thorough breakdown of the functionality in each section.

3. Ask about past issues. Are there areas of the app that have been particularly sensitive to bugs in the past? Issues you should keep an extra eye out for?

4. Talk to your manager or client. As with the steps in the “Timeline” section above, this can help set expectations. Let them know that you’re still getting up to speed with all of the sections of the software.

5. Make a list of questions. If there are specific sections or aspects of a feature you’re not sure about, you can attempt to find out. It might feel daunting to ask, but it’s better to test based on accurate information. (It can also come back to bite you if you don’t!)

Two testers reviewing bug reportsBug Reports

If you have experience in QA, you’re probably well-versed in writing bug reports. (If not, see our guide on Best Practices for Reporting Bugs.) However, the process can be optimized for regression testing.

When you’re in the middle of an extensive round of testing, it can slow you down to stop and make formal tickets for every little thing. But you also don’t want to skip reporting potential issues – no matter how small they are. So how do you proceed?

One of the best ways is to keep a simple list of notes as you test. That way, you can go back and create all of your bug tickets at the same time. If you happen to notice areas that you feel could benefit from user experience improvements, you can include those too. Regression testing is a great opportunity for assessing the overall user experience of the app or website.

That said, you’ll still want to call out any critical issues as you encounter them. The above method is great for bugs that seem to be “medium priority” or lower. (For more on this, see our guide on How to Prioritize Bug Fixes.) But if it’s a major issue – such as a highly reproducible crash – you’ll want to report it right away.

It’s also beneficial to remember that finding a bug in a new version doesn’t mean the bug itself is new. If you’re not positive, and you have time, it’s helpful to check old versions too. That way, you can see if the issue is reproducible there as well. This can also help determine whether the issue is on the back end (for example, with the API) or in the front end code.

How to Do Regression TestingQA tester learning how to do regression testing with a website and app

You can use the steps and advice above to follow best practices for regression testing. But if you’re trying to get into the field of QA and need experience first, remember that you can regression test anything! You don’t need to be working for a software company to perform your own regression testing.

For example, you can choose any app or website, and challenge yourself to go about regression testing it. While you won’t get paid for your time, the learning experience can pay off in the long run.

If you’re not sure where to start or what order to go in, don’t worry. You can start with smoke testing, and then go deeper in each area of every feature and section.

Regression Testing Estimation

Even for testing a single feature, creating estimates can be hard. Let alone when asked how long it will take to complete a full regression test of the entire app! There’s no way to guarantee that your estimate will be correct. However, you can follow some best practices to get as close as possible.

To learn more, check out our guide to Estimation Techniques.

Re-testing After Bug FixesDeveloper pointing to bug report with a checkmark

What happens if you find bugs and a new bug fix build is created? Do you have to redo all your previous regression testing? After all, the updates to the code could cause problems in areas you already checked. The answer is yes and no. It all depends on timing (see above), budget, and risk assessment.

In a perfect world, you would re-test the full software to be safe. But in reality, there just isn’t usually time for this. At minimum, QA should:

  • Test the bug fix itself, to make sure it’s working.
  • Regression test the section the bug had previously affected.
  • Smoke test major flows, such as login/signup.

Following those steps is a good balance of time, budget, and risk.

Automated Regression Testing

Automated testing is never going to be able to cover 100% of QA testing. However, one of the best areas it can help with is regression testing.

Software tester creating automated regression testing scripts on a laptop

It takes time to create new automation scripts. So it’s not always realistic to think that every new feature can be tested with automation before it’s released. But regression testing involves checking many of the same aspects every time. As a result, it’s a great candidate for automation.

For a good place to start, see the Top 5 Test Cases to Automate.

Regression Testing in Software Engineering

As you can see, regression testing plays a major role in software engineering. Will it find every possible bug in the software? Not necessarily — no one is infallible. But it’s the closest you can realistically get, and the most thorough type of software testing.