I never bought into the idea of DDT because it uses circular logic. That is to say that in every case I have found of people saying it's useful the consistent message is "It's useful because more cases mean it will catch more bugs.". In this type of logical reasoning it is claimed that A: "It's useful" because B: "more cases mean it will catch more bugs" even though B has not been proven and varies greatly depending on the area in the application that you are testing. In the way DDT is typically "sold" it is assumed that B is true therefore A is true. This is circular reasoning and is a logical fallacy.
Actually, I've never heard the argument for DDT as "do more cases, catch more bugs". Instead, a good DDT structured framework is not necessarily based upon generating a lot of test cases but more on separating the code that executes against the AUT from the data that the code uses. For example, I want to enter data into a form on screen. The code for that form is pretty static and basic, no matter what the data is. So, I abstract the code for the form to make all data entry fields and selections parameterized variables that are sourced in data.
Now, if I want to run a test case against that form, I simply create a data record for that form and boom. One test case is done. Want another test case? Don't need to touch the code at all, just add another data record.
Now... if done well, this isn't just looping through the form and populating data. That code that "drives" that form is called from other framework structures. So, I'm testing a use case, for example, that enters data in that form and then does some sort of operation based upon what was entered. So, my data includes not just the data on the form but other steps and processes, button clicks, validations, etc., that drive the whole use case.
Build your framework properly, and when it comes to adding another test case, the amount of actual code you need to write is reduced and most of your test case construction is simply adding data records. I had a framework at that company that did POS applications where I had code that applied payments, selected tickets, verified transaction journals, etc.... all modularized code. Everytime I wanted to add a new test case, I didn't need to touch a line of code, I simply went into my database, added a few data records, and now my project has expanded by a test case.
Sure, the end result is more test cases... but the true aim of DDT is to speed up the ability to add a test case and even make it so that a non-programmer tester can add a new test case by the simple task of data entry.
Little confused by that last response. You say you haven't heard an argument for more cases catching more bugs, but then all of this is to simplify the addition of more cases. I have to ask - if more cases doesn't mean more bugs then why is it so important to add more cases?
Ryan_Moran wrote:
Little confused by that last response. You say you haven't heard an argument for more cases catching more bugs, but then all of this is to simplify the addition of more cases. I have to ask - if more cases doesn't mean more bugs then why is it so important to add more cases?
You claim that the argument for DDT is "more test cases = more bugs". My explanation is intended to show that no, the argument for DDT is not that, it is that DDT allows for a more efficient way of adding test cases to a test suite.
ALWAYS, there is a decision making process, even in manual testing, "Is this test case worth running?" I'm not just going to add a test case because I can. I have to decide risk, ROI, code coverage, etc... once I determine that it's worth running, I want to make it as EASY as possible to add it to my automated test suite. DDT is the way to do that.
See Colin_McCrae's recent post, describing their process. It's not about creating more test cases for the sake of creating more test cases, it's about efficiency in adding to a test suite.
tristaanogre wrote:
See Colin_McCrae's recent post, describing their process. It's not about creating more test cases for the sake of creating more test cases, it's about efficiency in adding to a test suite.
AND it's about tracability (validation) of the testruns.
tristaanogre wrote:Actually, I've never heard the argument for DDT as "do more cases, catch more bugs". Instead, a good DDT structured framework is not necessarily based upon generating a lot of test cases but more on separating the code that executes against the AUT from the data that the code uses. For example, I want to enter data into a form on screen. The code for that form is pretty static and basic, no matter what the data is. So, I abstract the code for the form to make all data entry fields and selections parameterized variables that are sourced in data.
Now, if I want to run a test case against that form, I simply create a data record for that form and boom. One test case is done. Want another test case? Don't need to touch the code at all, just add another data record.
Now... if done well, this isn't just looping through the form and populating data. That code that "drives" that form is called from other framework structures. So, I'm testing a use case, for example, that enters data in that form and then does some sort of operation based upon what was entered. So, my data includes not just the data on the form but other steps and processes, button clicks, validations, etc., that drive the whole use case.
Build your framework properly, and when it comes to adding another test case, the amount of actual code you need to write is reduced and most of your test case construction is simply adding data records. I had a framework at that company that did POS applications where I had code that applied payments, selected tickets, verified transaction journals, etc.... all modularized code. Everytime I wanted to add a new test case, I didn't need to touch a line of code, I simply went into my database, added a few data records, and now my project has expanded by a test case.
Sure, the end result is more test cases... but the true aim of DDT is to speed up the ability to add a test case and even make it so that a non-programmer tester can add a new test case by the simple task of data entry.
tristaanogre That's DDT implemented at it's best
