SmartBear Talks | Effective Regression Testing in Agile - Gjore Zaharchev

[TanyaYatskovska]

Regressions testing is one of the important software testing types. It requires a lot of time to create tests, and twice or thrice as much time to execute them. The situation with regression testing becomes worse if you are trying to do regression tests during the development process. Thanks Agile! In this episode, a software quality expert, Gjore Zaharchev, will share with us his approach to regression testing that helped him significantly decrease the execution time.

Gjore Zaharchev

Gjore Zaharchev is Agile Evangelist and Heuristic Testing Fighter with more than 12 years of experience in automated, manual and also performance software testing for various domains and clients. During this period, Gjore has led and managed QA people and QA teams from different locations in Europe and the USA. He recognizes testers as people with various problem-solving skills and an engineering mindset and believes that software testers are more than mere numbers to clients. Currently, he is working at Seavus, with an official title of a Quality Assurance Coordinator responsible for the Software Testing Team. In addition, he is an active speaker on several conferences and events in Europe and Testing Coach at SEDC Academy in Skopje.

Any comments/questions/thoughts on the suggested approach to regression testing? We will be happy to discuss them - just leave a comment below.

Transcript:

Tanya

Hi SmartBear Community! Please meet today Gjore Zacharchev. Gjore, you're a big fan of different testing processes. I've watched several of your presentations, and they were so exciting! Could you please start with a short introduction of yourself to the SmartBear Community?

Gjore

Yeah sure, so, as I said my name is Gjore Zacharchev, I’ve been in the software testing area for more than 12 years. Currently, I'm working with Seavus in Macedonia, it's actually a multinational company. I'm responsible for the QA team here in Seavus. So, every day, I meet different demands and different approaches to testing, so in order to solve some of these things, I first need to understand them and then find a way to improve them. My presentation at SQA Days was about how to improve regression testing and to speed up the process in the Agile era.

Tanya

If we are talking about regression testing, let's give some short overview of how you set it up in your company.

Gjore

We are a very huge company, and we have so many projects, so many different approaches. We're trying to find the most efficient way of doing regression testing for a particular project. For the majority of the projects, we are trying to implement this regression testing heuristic called RCRCRC which stands for: recent, core, risky, configurations sensitive, repaired, chronic. Because we don't have a lot of time to do regression testing, we are trying to speed up this process by sorting all of our tests according to some criteria, based on these heuristics. These six groups can be also divided into two subgroups or two major groups: dynamic and static tests. Dynamic is something that is dynamically changed every sprint.

Tanya

As I understand, you are using different methodologies. So, where is it more difficult to do regression testing: in Agile, Waterfall, or AgiFall?

Gjore

Regression testing is difficult to be done in the Agile world, because we are getting late deliveries, we don't have to enough time to do testing. In Waterfall, it's pretty easy because you have dedicated time to do regression testing, you have all the resources, but Agile is more demanding. And, we are trying to avoid Water-Scrum-Fall.

We have very limited time to do regression testing, so we need to find a better way, a faster way, that is we first define criteria for regression testing, and this is the most important part.

Tanya

My colleagues were at SQA Days and they were really impressed by your speech where you said that you reinvented regression testing. What was the purpose of that, and what did you change?

Gjore

So, basically, when I came to this company, the process of doing regression testing was using the retest-all technique. People tested some areas of the application that were not important at all. It took a lot of time, and, sometimes, people were behind the schedule. So, my partner of reinventing the process was inventing criteria for testcase prioritization. And, this was the reinvented process. If you have that, it means you can limit your tests very reasonably. Some tests can be run even manually for several hours instead of days or weeks.

Tanya

Is it difficult to create such criteria?

Gjore

The criteria look very general, but it don’t have to be. First of all, you need to know the project, you need to know the application, the system. And, if you know this, it's pretty easy to group everything into six criteria, into six areas which will wait around to be changed, as I mentioned, three of them will be changed every sprint.

We have a lot of people who are beginners, and it was also pretty easy for them, it was better for them to define these criteria instead of retest-all which was time-consuming.

Tanya

What benefits did you get? How can you evaluate these improvements?

Gjore

The benefit first of all is that we can achieve regression testing within several hours because usually we were postponing regression testing to the next sprint. While developers were doing the development for the next sprint, people were moving regression testing into this sprint, which was pretty problematic because people were not attending planning meetings; they didn't attend daily meetings, so they were losing crucial information.

The second thing is reducing the time because we can do better testing during the entire period. We can do better regular testing of the product, so we can improve the quality even more.  In testing environments, we have deployments several times per day, so we can now run the automated regression tests several times after every deployment.

If we have automated tests at that point, there is no need to run manual tests.  Maybe, only for some specific cases that cannot be automated, or they are very difficult or very expensive to automate. So, we are going from manual to more than 80% of automated testing.

Tanya

Thanks a lot! That is a very interesting approach. SmartBear Community, it will be interesting to learn your opinion about this approach, so, please leave comments or maybe even ask Gjore additional questions. We'll continue talking about this approach in the comments. Thanks for joining us today! Thank you Gjore!

Gjore

Thank you, bye!