Forum Discussion

TanyaYatskovska's avatar
TanyaYatskovska
SmartBear Alumni (Retired)
6 years ago

The Secrets of Successful Collaboration - BDD

Hi our BDD TestComplete users, I want to share with you a great webinar presented by Seb Rose, developer, trainer, coach and author of "Discovery - Explore behavior using examples". Seb reveals the...
BDD
  • sebrose's avatar
    sebrose
    6 years ago

    Hi Alex,

     

    First, BDD is focussed on agreeing and documenting behaviours, not testing. This may be why you don't see BDD implementations that support tree-like expandable structures.

     

    In all but the most trivial systems, exhaustive path coverage is unattainable. Instead, metaphors such as the test automation pyramid were created to help us achieve confidence in a sustainable way.

    • Narrow tests (at the bottom of the pyramid) give us confidence that the code satisfies the detailed requirements.
    • Further up the pyramid there are tests that exercise interactions between a small number of components. This gives us confidence that the protocol/contract between the components has been correctly interpreted/implemented.
    • Finally, at the top of the pyramid, there are a few tests that exercise the whole of the system. These should be thought of as smoke/deployment tests, giving us confidence that the system has been deployed correctly, is talking to the right end points, has the correct authentication tokens for the environment etc. Broadly, that it "hangs together".

    Combining these different types of test together can give us confidence that the system behaves as requested -- especially if the test doubles used to ensure isolation for the narrow and protocol tests are covered by contract tests. Other testing is also needed: exploratory, penetration, load etc. I've written a couple of blogs that partially cover these topics with this here and here.

     

    When writing Journey Scenarios, we need to ensure that they are written using business language. They should express the intent of what the actor interacting with the system is trying to achieve, rather than the mechanics of how they will achieve it. The technical details will be wrapped up in the automation code (step definitions) rather than being exposed to readers of the specification. However, the automation code is also used (and validated) in the narrow scenarios that guided the original implementation.

     

    To your final point, about supporting other stakeholders (e.g. Sales & Marketing), I have seen implementations where a single feature had several feature files, each of which targeted a different audience (in a recent example they were Sales and Actuarial). This is hugely beneficial for communication, but is definitely not motivated by a desire for exhaustive test coverage.

     

    In summary:

    1. BDD is not focused on testing
    2. Exhaustive path coverage is unattainable (so we lean on the test automation pyramid)
    3. Journey scenarios that use business language can be understood by (and are interesting to) less technical colleagues
    4. Documentation can (and should) be tailored to the specific stakeholders with whom you are collaborating

    Thanks

    Seb

AlexKaras's avatar
AlexKaras
Champion Level 3

Hi Seb,

 

Thank you a lot for the followup comments.

Yes, I absolutely agree as for Discovery and Formulation and this is what was really useful at least for me.

I also agree as for the shared understanding expressed using common language.

 

> The majority of the behaviours that we describe in BDD scenarios are quite narrow [bolded by Alex]

This is exactly what I meant from practical point of view of tests automation practitioneer: simple narrow scenarios may contain all details required for successful automation.

However wide "Jorney" scenarios for end-to-end tests usually require a lot of details for successful automation. With this level of details, the scenario will be absolutely irrelevant for business because it naturally does not need this level of details.

On the contrary, automation of the high-level business scenario will result in that a lot of cases that can happen during test flow will be handled by test author according to his own understanding without been documented in scenario. As a result, we will not get the test that, by definition, verifies the requirement (even incorrect one!). Instead, we will get something like automated exploratory scenario validation done from the subject matter expert's point of view. And this point of view may differ from the (possibly incorrectly formulated) requirement.

And, unfortunately, I did not hear about BDD implementations that support tree-like expandable structure when at the root we can have a business statement and expand it down to the level required by Sales, Marketing, Development, Testing and so on.

 

This is what I meant  in my initial reply to this thread.

 

 

sebrose's avatar
sebrose
SmartBear Alumni (Retired)
6 years ago

Hi Alex,

 

First, BDD is focussed on agreeing and documenting behaviours, not testing. This may be why you don't see BDD implementations that support tree-like expandable structures.

 

In all but the most trivial systems, exhaustive path coverage is unattainable. Instead, metaphors such as the test automation pyramid were created to help us achieve confidence in a sustainable way.

  • Narrow tests (at the bottom of the pyramid) give us confidence that the code satisfies the detailed requirements.
  • Further up the pyramid there are tests that exercise interactions between a small number of components. This gives us confidence that the protocol/contract between the components has been correctly interpreted/implemented.
  • Finally, at the top of the pyramid, there are a few tests that exercise the whole of the system. These should be thought of as smoke/deployment tests, giving us confidence that the system has been deployed correctly, is talking to the right end points, has the correct authentication tokens for the environment etc. Broadly, that it "hangs together".

Combining these different types of test together can give us confidence that the system behaves as requested -- especially if the test doubles used to ensure isolation for the narrow and protocol tests are covered by contract tests. Other testing is also needed: exploratory, penetration, load etc. I've written a couple of blogs that partially cover these topics with this here and here.

 

When writing Journey Scenarios, we need to ensure that they are written using business language. They should express the intent of what the actor interacting with the system is trying to achieve, rather than the mechanics of how they will achieve it. The technical details will be wrapped up in the automation code (step definitions) rather than being exposed to readers of the specification. However, the automation code is also used (and validated) in the narrow scenarios that guided the original implementation.

 

To your final point, about supporting other stakeholders (e.g. Sales & Marketing), I have seen implementations where a single feature had several feature files, each of which targeted a different audience (in a recent example they were Sales and Actuarial). This is hugely beneficial for communication, but is definitely not motivated by a desire for exhaustive test coverage.

 

In summary:

  1. BDD is not focused on testing
  2. Exhaustive path coverage is unattainable (so we lean on the test automation pyramid)
  3. Journey scenarios that use business language can be understood by (and are interesting to) less technical colleagues
  4. Documentation can (and should) be tailored to the specific stakeholders with whom you are collaborating

Thanks

Seb

  • AlexKaras's avatar
    AlexKaras
    Champion Level 3
    6 years ago

    Hi Seb,

     

    Thank you again for your detailed comments, much appreciated indeed.

    > BDD is not focused on testing
    This is a key point.
    And I think that this is the thing that must be understood by both, those who do initial 'business-faced' part of work and those who do a technical one.