Day 3.2 : Difference between regression, sanity and smoke tests

-

 

Smoke Testing

Smoke testing is an approach which is usually carried out during the initial development stages of the Software Development Life Cycle(SDLC) to make sure that the core functionalities of a program are working fine without any issues. It is executed before any detailed functional tests are done on the software.

The main intent of smoke testing is not to perform deep testing but to verify that the core or main functionalities of the program or the software are working fine. Smoke testing aims to reject a badly broken build in the initial stage so that the testing team does not waste time in installing & testing the software application.

Smoke testing is also called as Build Verification Test.

Let’s see a simple example where you are given an email application to test. The important functions would be logging in to the email application, composing an email and sending it, right? And, in case the email is not sent, does it make any sense to test other functionalities like drafts, deleted messages, archives, etc? This means you will have to drop the build without further validation. This is called Smoke testing.

The main focus of smoke testing is to test the critical areas and not the complete application.

When to perform Smoke Testing

  • When developers provide a fresh build to the QA team. A fresh build here means when the build has new changes made by the developers.
  • When a new module is added to the existing functionality.

Automation & Smoke Testing:

Usually this is the type of testing that is executed before actual automation test cases can run. For organizations that have continuous testing built in, smoke testing is equivalent to successful installation of the build for running test cases or execution of the first test case. So, this is not a type of testing that is deliberately automated but if test automation is put into place, the test automation can only run successfully once the software has passed smoke testing. Or otherwise, the first test case that executes might fail.

Sanity Testing

Sanity testing is a kind of testing performed to check whether a software product is working correctly when a new module or functionality gets implemented to an existing product. Sanity testing is a software testing technique which does a quick evaluation of the quality of the software release to determine whether it is eligible for further rounds of testing or not.

Sanity testing is usually performed after receiving a fairly stable software build or sometimes when a software build might have undergone minor changes in the code or functionality. It decides if end to end testing of a software product shall be carried out further or not.

Sanity testing is also a Surface Level Testing which helps in deciding if the software build is good enough to pass it to the next level of testing.

Why perform Sanity Testing

  • To verify and validate the conformity of newly added functionalities and features in existing code.
  • To ensure that the introduced changes do not affect other existing functionalities of the product.
  • To decide further testing can be carried forward or not.

When to perform Sanity Testing

  • Build is received after many regressions or if there is a minor change in the code.
  • The build is received after bug fixing.
  • Just before the deployment on production.

Automation & Sanity Testing:

Considering, Sanity Testing is considered as a subset of regression testing, these are the test cases that can be automated. A recommended approach is to execute these test cases before running the complete regression test suite. The benefit is that if there are any errors in the sanity test cases, then errors can be reported sooner rather than later.

There are companies that have benefited from automating their sanity test cases. Here is a case study where Freshworks – a feature packed product suite for businesses of all sizes – was able to automate 100% of their sanity test cases in 90 days using Testsigma: Testsigma+Freshworks – A case study

Regression Testing

Regression testing is the verification of “bug fixes or any changes in the requirement” and making sure they are not affecting other functionalities of the application. Regression testing is effective on automation and usually performed after some modifications have been made in the software build after requirement changes or bug fixes.

Once Sanity testing of the changed functionality is completed, all the impacted features of the application require complete testing. This is called regression testing.

Whenever bug fixes are done in the existing software, some test scenarios need to be executed, to verify the bug fixes. In addition to these, the QA team also has to check the impacted areas, based on the code changes. In regression testing, all those test scenarios will have to be executed, to take care of related functionalities.

When to perform Regression Testing

  • After Code modification according to the required changes
  • After some new features are added to the application
  • After some bug fixes are incorporated into the build

Automation & Regression Testing:

Regression test cases are actually the ideal test cases for automaton. Usually, when an organization starts automation, these are the test cases that are automated first. If regression testing is an activity that is taking a lot of time for your testers and the same test cases are repeated multiple times then it is time that you start thinking of automation too.

If you are looking for an automated regression testing tool that can help you get started on your automation journey, then you also need to ensure that you choose the right tool. A tool that can also provide you ROI on the efforts invested. We have the guide that can help you there:

https://testsigma.com/blog/10-points-to-help-you-choose-the-right-test-automation-tool/

Differences Between Smoke vs Sanity vs Regression Testing

Smoke TestingSanity TestingRegression Testing
Performed on initial buildsPerformed on stable buildsPerformed on stable builds
To test the stability of new buildTo test the stability of new functionality or code changes in the existing buildTo test the functionality of all affected areas after new functionality/code changes in the existing build
Covers end to end basic functionalitiesCovers certain modules, in which code changes have been madeCovers detailed testing targeting all the affected areas after new functionalities are added
Executed by testers & sometimes also by developersExecuted by testersExecuted by testers, mostly via automation
A part of basic testingA part of regression testingRegression Testing is a super set of Smoke and Sanity Testing
Done usually every time there is a new buildPlanned when there is not enough time for in-depth testingUsually performed, when testers have enough time

Key Points

  • Smoke and Sanity testing help the QA team save time by quickly testing to make sure if an application is working properly or not. Also, it ensures that the product is eligible for further testing. Whereas Regression testing helps enhance the confidence about the software quality after a particular change. Especially, that the code changes are not affecting related areas.

  • Smoke Testing is done by both the dev team or by the QA team and can be taken as a subset of rigorous testing. Whereas both Sanity & Regression testing are done only by the QA team. Also, Sanity testing can be considered as a subset of acceptance testing.

  • Smoke testing is executed at the initial stage of SDLC, to check the core functionalities of an application. Whereas Sanity & Regression testing are done at the final stage of SDLC, to check the main functionalities of an application.

  • As per the requirement of testing & time availability, the QA team may have to execute Sanity, Smoke & Regression tests on their software build. In such cases, Smoke tests are executed first, followed by Sanity Testing & then based on time availability regression testing is planned.

In practice, all QA teams need to do Smoke, Sanity and Regression testing. All of these testing types have a pre-defined number of test cases that get executed multiple times. This repetitive execution also makes them an ideal candidate for test automation. When looking for automation, you are recommended to use a tool that provides you ROI on automation from the initial stages. Testsigma is one such tool.

Comments

Post a Comment

Popular posts from this blog

Day 2.2 : What is STLC

-
Image
What is STLC STLC stands for Software Testing Life Cycle . STLC is a sequence of different activities performed by the testing team to ensure the quality of the software or the product. STLC is an integral part of Software Development Life Cycle (SDLC). But, STLC deals only with the testing phases. STLC starts as soon as requirements are defined or SRD (Software Requirement Document) is shared by stakeholders. STLC provides a step-by-step process to ensure quality software. In the early stage of STLC, while the software or the product is developing, the tester can analyze and define the scope of testing, entry and exit criteria and also the Test Cases. It helps to reduce the test cycle time along with better quality. As soon as the development phase is over, the testers are ready with test cases and start with execution. This helps to find bugs in the initial phase. STLC Phases STLC has the following different phases but it is not mandatory to follow all phases. Phases are dependent on...
Read more

Day 1

-
Day 1:  Met my lovely classmates  and  our week 1 to 3 teacher Galen Yang. He has a 10 years experience in testing. Task 1 : Define the following          a. What is a client and a server? In testing, client is the one sending the request going to the server/web application. A server is where the web/ desktop application is residing or installed.         b. What is cloud based software? A cloud base software is basically  an application residing or installed  on the cloud. Example of cloud : Azure , AWS , Google        c. What are the different environments in a software development team? Usually there are 3 environment in software  develop.  1. Dev Environment - This environment is solely use by developers. This is were the codes is saved/residing. 2. test  Environment - this environment is solely use by Tester, where they  going to test the a...
Read more

Day 2.1 : What is SDLC?

-
Image
What is SDLC?  Software Development Life Cycle (SDLC) is a process used by the software industry to design, develop and test high quality softwares. The SDLC aims to produce a high-quality software that meets or exceeds customer expectations, reaches completion within times and cost estimates. SDLC is the acronym of Software Development Life Cycle. It is also called as Software Development Process. SDLC is a framework defining tasks performed at each step in the software development process. ISO/IEC 12207 is an international standard for software life-cycle processes. It aims to be the standard that defines all the tasks required for developing and maintaining software. SDLC is a process followed for a software project, within a software organisation. It consists of a detailed plan describing how to develop, maintain, replace and alter or enhance specific software. The life cycle defines a methodology for improving the quality of software and the overall development process. The ...
Read more