Wintertainment 2020 - Day 5: Understanding the Benefits of Test Automation and BDD

sebrose · ‎12-11-2020

Hi SmartBear Community! Today's discussion will be around test automation and BDD benefits.

Check out the article below that explains the subject perfectly, and answer the questions at the end of the article. We will be happy to learn what you think!

The most active participants who provide in-depth and insightful replies will be rewarded 🎁

See participation rules and event schedule.

Understanding the Benefits of Test Automation and BDD

Teams have been automating tests for as long as software has existed. With the increased adoption of continuous integration and delivery, automated tests are becoming essential. Behaviour-driven development (BDD) is a much newer approach to developing software that grew out of Agile and test-driven development.

Most teams/organisations can benefit from test automation and BDD, but BDD adoption is much more than simply automating tests.

Test automation

It’s important to define a few commonly used terms.

Test: An activity that gives you information about how a system is performing.
Automation: The ability to carry out an activity without human involvement.
Test automation: Running (and sometimes interpreting) tests without human involvement.

From these definitions you can see that test automation is a very broad term. It encompasses any activity that generates information about a system which can be run automatically.

Behaviour-driven development

BDD is a specific set of practices that lead to improved outcomes for Agile teams and their organisations. The three core practices of BDD are:

Discovery: Create a shared understanding of the requirements through collaboration, typically achieved through a structured conversation centred on rules and examples.
Formulation: Examples of system behaviour are documented using business terminology.
Automation: Implementation design is guided by the automation, requiring the full participation of the delivery team before the corresponding implementation code is written.

As you can see, BDD has a much more prescriptive definition than test automation. While it does lead to the creation of some automated tests, this is not its primary goal, and the focus of the tests is usually limited to the functional behaviour of the system.

Comparison

The primary goal of test automation is to reduce the dependency on humans needing to execute tests. BDD, on the other hand, has grander aspirations – to reduce misunderstandings and rework by requiring collaborative discovery and formulation BEFORE any implementation code is written. Additionally, since the automation of the business-readable specification is inextricable from the design of the implementation, it needs to be carried out by the delivery team, not a separate automation team.

Consequently, BDD has broader impact on the team/organisation. It requires early and frequent collaboration where it did not exist before, and the full participation of the development team in automation. Test automation is less prescriptive and requires little organisational change.

The focus of BDD is specifying the functional behaviour of the system – non-functional requirements are rarely covered. Test automation is applicable to any test domain (functional or non-functional) that is amenable to automation.

To summarise:

	Test automation	BDD
Description	Any activity that results in automated tests.	A set of practices that helps organisations refine requirements and deliver software collaboratively.
Primary goal	Reduce the dependency on humans to execute the tests.	Reduce misunderstandings and rework by reaching a shared understanding of a story before development begins.
Organisational impact	Little organisational impact beyond selection of relevant tools that target selected test domains (e.g. penetration, load etc.). Automated during/after implementation.	Broad impact through increased early collaboration, a focus on using business terminology, and automation by the development team BEFORE implementation.

Conclusion

Test automation and BDD have different goals and require different patterns of collaboration across the organisation. Adopting BDD does contribute to test automation, but that is not its primary goal.

Test automation and BDD are both valuable, but it is a mistake to think of them as the same thing – or even closely related. Most organisations would benefit greatly from applying both approaches.

What do you think? Comment below!

We'd love to hear about your experience with test automation and BDD!

Do you use BDD or test automation in your organisation? Have you experienced any tensions between them? Are you concerned about possible duplication of automated tests?
Have you used the Given/When/Then keywords to write automated test scripts without first applying the collaborative BDD practices of Discovery and Formulation?
When do you write your automated tests? Before or after the implementation code is written?

HimanshuTayal · ‎12-11-2020

A very Yes!!!

BDD is current demand as many organizations and team are following Agile model, this helps a lot in time in Automation testing as PO can also write feature file 🙂

1. Do you use BDD or test automation in your organisation? Have you experienced any tensions between them? Are you concerned about possible duplication of automated tests?

Yes i do use BDD for Automating test cases with in team it saves time sometimes coz we can ask functional team/POs to write down the feature file for us.

2. Have you used the Given/When/Then keywords to write automated test scripts without first applying the collaborative BDD practices of Discovery and Formulation?

3. When do you write your automated tests? Before or after the implementation code is written?

It depends if we have clear picture of Functionality then we can write it before the implementation of code. But as an individual i am writing after the implementation.

Marsha_R · ‎12-11-2020

1) In my last organization, we managed to implement Agile pretty well, and we tried BDD but it was a struggle. A couple of us caught on right away but others had a hard time. It was still a work in progress when I left there. My current place exists in a time warp so that any change is very very very sloooooow. It's taken us over a year to get sprints into place and the workflow for tickets is still a WIP.

2) I haven't used this process.

3) At the last place, I could start writing tests when development was happening because I had access to the correct environment. In the current one, I may or may not have access to the environment and I certainly don't have access to the documentation to even have a clue where to start.

DSmachineWorld · ‎12-11-2020

I've tried using BDD on one project, it seems like a very helpful practice. I used the coloured index card method to gather information. I would like to repeat it in the future, when sitting down in a room with coloured index cards is again possible.

As for writing the BDD testing framework, I don't think you should start that without doing the business requirements gathering first in the BDD session. If you don't do that, I think you will struggle with the correct granularity level of the tests you are trying to write.

SiKing · ‎12-11-2020

@sebrose wrote:

BDD, on the other hand, has grander aspirations – to reduce misunderstandings and rework by requiring collaborative discovery and formulation BEFORE any implementation code is written.

This!!!

I have recently (in the last year) gotten into BDD tests; I am using Serenity BDD. However, without participation of the rest of the team - not just development, but business - it is only extra overhead for me. Currently we write test after development, often after deployment 😱, and so using BDD makes it just an interesting exercise for me.

Hoping to improve this in the coming year.

yhyman · ‎12-14-2020

I enjoyed reading this article. I like the way it distinguishes between the goals of automation and BDD. We got as far as the Formulation stage for BDD and have even found that to be helpful for new people starting on a project. What is especially helpful is using a Gherkin kind of Given/When/Then format. Unfortunately, we are struggling to get to the automation piece due to the small team size, time constraints and skill level. Any advice for how to move forward?

sebrose · ‎12-14-2020

@yhyman Automation *should* save you time, by providing regular feedback and removing the need to find team members to run regression tests. There's a free course that teaches BDD with Cucumber at school.cucumber.io - which would be a great place to start.