Forum Discussion
jsc wrote:Team
...
All of our testitems can be run on their own. No testitem depends on another. The testitems should take less than 4-5 minutes (there are some legacy tests, that run stable but take way too long).
Excellent stuff. Thanks for all your feedback. I'll probably ask several questions from the one post.
How are all testitems designed to run on their own and not depend on another? If a record is added in a testitem and not removed how can the test be run again without creating a duplicate? If there are dependencies (i.e. supplier / product / purchase / receive / customer / sale / receivable...) how can individual testitems be descrete and reusable and run on their own?
How are all testitems designed to run on their own and not depend on another?
- they all start with a defined state
- there are 2 defined starting-states in our testitems: either a new and empty database is created at the start of the testitem or an existing database is restored
- often we import specified data (this is possible in our software, e.g. from an excel file that is also under version control)
- afterwards they all start the browser and log in
If a record is added in a testitem and not removed how can the test be run again without creating a duplicate?
- see above
- additionally we do some preparing work at the start of the project suite (e.g. installing the program, deleting all existing databases, terminating some processes...) (setup: ~9 minutes)
- additionally we have a onStartTest routine running at the start of each testitem (terminating browser, terminating processes,...) (onStartTest: ~2-5 seconds)
- we also have a onStopTest-Routine (~1-3 seconds) and a teardown project that clears all stuff at the end of the project suite (~20-30 seconds)
- no testitem is run twice. there is no need for that in our case. and if so, we would take care about a defined state at the start of the testitem (see above)
If there are dependencies (i.e. supplier / product / purchase / receive / customer / sale / receivable...)
how can individual testitems be descrete and reusable and run on their own?
- I do not get what dependencies this could be, but I try to guess.
- e.g. we have a database that we restore where a webservice-connector or other connectors are already defined
- we have surrounding testenvironment that provides data (some vm sontaining ppm-systems, excel files, webservices,...) that is either under version control or must not be changed at all.
- is it this what you meant?
Additionally we make sure that a failed testitem does not at all affect following testitems.
We therefore wrote our own Runner.nextTestitem(...) routine that skips the current testitem and starts the next. This routine is called when some inproper state is detected (restoring database failed, timeout is over,...). This routine combined with the onStartTest and onStopTest routines guarantee us that this is working perfectly well.
- baxatob9 years agoCommunity Hero
We were able to make the developers take responsibility for manual testing, code reviews, unit tests and integration tests. Now the last step is coming that the developers also should do the automated regression testing.
No, the last step should be - to make the developers take responsibility for making the coffee for testers :)
(I'm just kidding. Thank you for your detailed feedback)
- jsc9 years agoRegular Contributor
we do it like this (Jscript)
function startTask(taskname, waitTime)
{
Log.AppendFolder("startTask: " + taskname);
Log.Message("starting task '" + taskname + "'");
var oShell = new ActiveXObject("WSCript.shell");
oShell.run('NET START "' + taskname + '"', "1", true);
Log.Message("trying to start task '" + taskname + "'");
Delay(waitTime);
Log.Message("waited for " + waitTime + " ms");Log.PopLogFolder();
}Stopping a task is the same. But we combine it with checking afterwards if the corresponding process does still exist. If the process still exists it is just killed. (script as well)
- william_roe9 years agoSuper Contributor
jsc wrote:If there are dependencies (i.e. supplier / product / purchase / receive / customer / sale / receivable...)
how can individual testitems be descrete and reusable and run on their own?
- I do not get what dependencies this could be, but I try to guess.
- e.g. we have a database that we restore where a webservice-connector or other connectors are already defined- we have surrounding testenvironment that provides data (some vm sontaining ppm-systems, excel files, webservices,...) that is either under version control or must not be changed at all.
- is it this what you meant?l.
Examples of dependencies our system:
- Product(s) cannot be created with Supplier(s)
- Purchase Order(s) cannot be created without Product(s)
- Receipts cannot be created without Purchase Order(s)
- Sale(s) cannot be created without Inventory (and Customers)
- Receivable(s) cannot be created without Sale(s)
I understand your approach but have a question. How are you restoring your database while the application pool (IIS) is running? I regularly restore the database to certain pre-defined states (i.e. pre-sales) when there is a failure but this requires stopping the web server application pool and killing an application process before a restore can be performed.
Are you a Microsoft shop? We use IIS and SQL Server (donning fire suit).
- william_roe9 years agoSuper Contributor
baxatob wrote:We were able to make the developers take responsibility for manual testing, code reviews, unit tests and integration tests. Now the last step is coming that the developers also should do the automated regression testing.
No, the last step should be - to make the developers take responsibility for making the coffee for testers :)
(I'm just kidding. Thank you for your detailed feedback)
The better the automated testing the lazier developers become :smileytongue:
- jsc9 years agoRegular Contributor
william_roe wrote:- Product(s) cannot be created with Supplier(s)
- Purchase Order(s) cannot be created without Product(s)
- Receipts cannot be created without Purchase Order(s)
- Sale(s) cannot be created without Inventory (and Customers)
- Receivable(s) cannot be created without Sale(s)
I understand your approach but have a question. How are you restoring your database while the application pool (IIS) is running? I regularly restore the database to certain pre-defined states (i.e. pre-sales) when there is a failure but this requires stopping the web server application pool and killing an application process before a restore can be performed.
Are you a Microsoft shop? We use IIS and SQL Server (donning fire suit).
I think we do not have such dependencies, but I would (or at least try to) integrate them in a stable state in our testenvironment.
We use a postgreSQL databases and apacache Tomcat in our web application.
The restoring databases is easy. The service is stopped and started via script as well as the Tomcat service. So we make sure that all services are started properly when login starts.
(in other departments we do similar stuff with MS SQL Server (no problem at all) and Oracle DMBS (bigger and therefore slower to restore)).
No, we are no webshop, we create some sort of ppm-software. (I am not sure if it is allowed to post links to our software here, so I do not...)
- william_roe9 years agoSuper Contributor
jsc wrote:
william_roe wrote:- Product(s) cannot be created with Supplier(s)
- Purchase Order(s) cannot be created without Product(s)
- Receipts cannot be created without Purchase Order(s)
- Sale(s) cannot be created without Inventory (and Customers)
- Receivable(s) cannot be created without Sale(s)
I understand your approach but have a question. How are you restoring your database while the application pool (IIS) is running? I regularly restore the database to certain pre-defined states (i.e. pre-sales) when there is a failure but this requires stopping the web server application pool and killing an application process before a restore can be performed.
Are you a Microsoft shop? We use IIS and SQL Server (donning fire suit).
.... So we make sure that all services are started properly when login starts.
(in other departments we do similar stuff with MS SQL Server (no problem at all) and Oracle DMBS (bigger and therefore slower to restore)).
Sounds as though our next step is to figure out how to start / stop IIS (via script) and how to terminate / launch applications via script.
GREAT DISCUSSIONS!!!