Ask a Question

SmartBear Talks | Run a Perfect Bug Hunting Event in DevOps - Szilard Szell

Repetitive testing in an Agile or DevOps team might become exhausting, and even boring. From time to time, the teams need to slow down and sharpen their axes to be able to continuously excel in their daily work. One opportunity is to run a bug hunting session, to measure the quality of the software, to learn about the product, and to improve the team spirit. Szilard Szell, DevOps and Test Automation Expert, is sharing how to excel in exploratory testing and make bug hunting events a crucial part of the team life and where this concept fits into Scaled Agile Framework and DevOps. Watch the interview with Szilard and share your experience after doing bug hunting in comments.



Szilard Szell

For the last 20 years, Szilard’s main goal has been to help people become addicted to Software Testing, and excel in it. He has a proven track record in large scale Telco DevOps Transformation Programs, in assessments, as a trainer and facilitator and coach in the Test Automation and Testing Processes area.


As a conference speaker, he is ready to share his view and knowledge with Testing Fellows, that is based on 20 years of industry experience and multitude of certifications held (he is an ISTQB Certified tester: Expert Level – Improving the Test Process, Agile Tester and Test Manager. Also owns DevOps DASA Foundation, SAFe SPC, IREB CPRE, Certified Scrum Master and Lean Six Sigma Green Belt certificates).


To give back to the Testing Community, Szilard is active in ISTQB as the Work Group lead of Processes Management and Compliance of ISTQB, and the former President of the Hungarian Testing Board, as well as active member of the Programme Committee of HUSTEF and UCAAT Confereces. He works as a Senior Consultant and 5G DevOps Transformation lead at Eficode.


Any comments/questions/thoughts? We will be happy to discuss them - just leave a comment below.




Today we are going to talk about bug hunting in DevOps - what it is, when it is better to start it, and what you can achieve. For this, we have invited a special guest, Szilard Szell – a DevOps test automation expert and well-known testing coach.


Let's start with the definition. How would you define bug hunting and why is it important nowadays?



Bug hunting, I would say, is more of a team building. It is more about having some fun in the testing team. But what is actually bug hunting? Bug hunting is when you do exploratory testing on the team level. You bring people together, you do a little competition with the goal of breaking your system, with the goal of learning about the behavior of your system.


This is a good way, and I learned this from Klaus Olsen and in 2013 during a conference in Helsinki. His first slide was ‘My goal is that you do bug hunting starting tomorrow.’ And, I loved that. I went back to my team, and I started to learn about bug hunting, and I started to apply it with the team, and it was fun.



So, what preparations should companies or testers make in order to go bug hunting?



Okay. As I said, bug hunting is a little bit of a team event. So, of course, you need to have your team together either in one location, or if you have multiple locations, you can even use WebEx or any kind of connection to have that.


You need to have small teams with three or four people, and they will be actually competing with each other.

You need to have test equipment for all of them, like your system, your software, your application, whatever is in your scope.


This is exploratory testing, it's actually a little bit of session based testing with an idea of what you want to learn, what you want to achieve, what features you are looking for, or what aspects of the product you are looking for.


And then, you prepare this by setting up the teams, you are setting the scope. It's also good to describe the known faults because we want people to find new faults. So, you make a list of all the known faults beforehand so that no one is ‘cheating’. Then you start hunting.


You can have multiple sessions. For example, if you have three teams with three equipment sets, then you have three sessions. After this, you have a round robin so everybody's working on every one of the different environments.


You have roughly 45-minute or 1-hour sessions when you let the team hunt for bugs, hunt for any strange things. As soon as anyone finds an issue, it's important that they are loudly stating to the others what they found. If you use WebEx, then you can put it in the chat describing very shortly what you found some with some screenshot, screen capture, log, whatever, explaining how you know it's a bug.


We used bicycle horns, so whenever there was a bug, people started to honk very loudly. Everybody came to find out what the noise was. We said, ‘Oh, we are just doing bug hunting!’ It's fun, it's not serious at all.



But, as a result, you are getting tons of issues, tons of bugs.



A lot, a lot. Actually, on one of the early sessions, we did it I think with eight teams in two locations for one hour two times, they came up with 96 bugs. Almost a hundred bugs. Now, roughly half of them were real issues.


There is also one guy who should walk around and check that it is really something. This is kind of a coach going around and checking that this is not a duplication.


At the end, you count what the team found, you are happy, you celebrate each other, and you then call for the winner. Typically, we have two prizes. You need to have a prize. In Hungary, it was always some beer or chocolate. It works well. It is just, you know, a small reward.


As a rule, what we called for is the team that found the biggest amount of bugs. And, the team that found the weirdest, the strangest bug. So, typically, we have these two prizes. The result is that you have these issues, and after the bug hunting event, you go through them again, and then you create bug reports for them. So, they are not just going to waste, you have to report them.


We had difficulties on some of the programs; management was asking what happened on that day why we have a big peak of incoming faults. We said, ‘We are doing bug hunting.’ And, they said very nicely, ‘OK, stop doing it.’


But, actually, it was very nice, and people liked it a lot, they appreciated the learning, they appreciated the team spirit, they liked actually what was happening, and they learned a lot about the system itself.



That sounds very fun and very productive at the same time. So many bugs, so many issues will appear in the issue tracking system. What is the next step? How to fit bug hunting into the existing DevOps process?



I would say bug hunting fits anywhere. It fits very nicely into the agile development - you should do it in every iteration. Maybe, it's a Friday afternoon after you do the demo, or before that, before you do the demonstration. It might be part of your sprint review. The retrospective part.


We did also extended it to the DevOps part when you do this together with your customers, with your end-users actually deploying your software in production and doing the bug hunting there.


Bug hunting can happen anywhere. It can happen within the iteration, at an early stage to measure and give feedback to your development team, to the cross-functional team, to show them how it works.


Bug hunting can be even done together with the product management to show them how the system behaves from the end-user point of view. So, I really feel this is about the DevOps values of collaboration, working together with your customers continuously giving feedback about the product and about your knowledge. So, bug hunting just fits very nicely into these values, as well.



Does it make sense to involve beta testers in bug hunting? I know that some companies invite beta testers which are not employees of the company, they may be even users.  



I think they are anyway doing that on their own. They are the best in killing our systems. They have magic hands for that. But, actually, it's a good idea to bring them in. You can actually then call it a hackathon, for example. Bring them together! Because bug hunting is really on not only finding faults, but it's about building your team as I said. So, it's a good team building event. And, doing that with beta testers you learn a lot from the end-user viewpoint which is I think extremely important.


Bug hunting is giving you a little bit of slack, giving you a little bit of a pause in that continuous rushing and running. It's just freeing up a lot of energy, and, that's, why I'm advising doing it on Friday. Learn from it and then go home with a big smile!



Okay, great, thanks a lot for this information! And, thanks a lot for your time. I'm sure that our community members will go bug hunting right after watching the video!



I hope so. You can find a lot of material on the Internet related to this one, so it's not hard to do. I really wish that you go bug hunting starting next week after watching this video!



So, Community go bug hunting, post your comments, we will be happy to continue the conversation! Thank you very much!



Thank you! Bye!

SmartBear Alumni (Retired)