who does your test automation - manual testers or programmer

[vthomeschoolmom]

I am asking if there are a lot of folks out there for whom this is considered an engineering function vs a manual tester's function. Thanks.

[jsc]

How are all testitems designed to run on their own and not depend on another?
- they all start with a defined state
- there are 2 defined starting-states in our testitems: either a new and empty database is created at the start of the testitem or an existing database is restored

- often we import specified data (this is possible in our software, e.g. from an excel file that is also under version control)
- afterwards they all start the browser and log in

If a record is added in a testitem and not removed how can the test be run again without creating a duplicate?
- see above
- additionally we do some preparing work at the start of the project suite (e.g. installing the program, deleting all existing databases, terminating some processes...) (setup: ~9 minutes)
- additionally we have a onStartTest routine running at the start of each testitem (terminating browser, terminating processes,...) (onStartTest: ~2-5 seconds)

- we also have a onStopTest-Routine (~1-3 seconds) and a teardown project that clears all stuff at the end of the project suite (~20-30 seconds)
- no testitem is run twice. there is no need for that in our case. and if so, we would take care about a defined state at the start of the testitem (see above)

If there are dependencies (i.e. supplier / product / purchase / receive / customer / sale / receivable...)
how can individual testitems be descrete and reusable and run on their own?
- I do not get what dependencies this could be, but I try to guess.
- e.g. we have a database that we restore where a webservice-connector or other connectors are already defined

- we have surrounding testenvironment that provides data (some vm sontaining ppm-systems, excel files, webservices,...) that is either under version control or must not be changed at all.
- is it this what you meant?

Additionally we make sure that a failed testitem does not at all affect following testitems.

We therefore wrote our own Runner.nextTestitem(...) routine that skips the current testitem and starts the next. This routine is called when some inproper state is detected (restoring database failed, timeout is over,...). This routine combined with the onStartTest and onStopTest routines guarantee us that this is working perfectly well.

[baxatob]

We were able to make the developers take responsibility for manual testing, code reviews, unit tests and integration tests.
Now the last step is coming that the developers also should do the automated regression testing.

No, the last step should be - to make the developers take responsibility for making the coffee for testers 🙂 

(I'm just kidding. Thank you for your detailed feedback)

[william_roe]

---

@jsc wrote:

If there are dependencies (i.e. supplier / product / purchase / receive / customer / sale / receivable...)
how can individual testitems be descrete and reusable and run on their own?
- I do not get what dependencies this could be, but I try to guess.
- e.g. we have a database that we restore where a webservice-connector or other connectors are already defined

- we have surrounding testenvironment that provides data (some vm sontaining ppm-systems, excel files, webservices,...) that is either under version control or must not be changed at all.
- is it this what you meant?

l.

---

Examples of dependencies our system:

• Product(s) cannot be created with Supplier(s)
• Purchase Order(s) cannot be created without Product(s)
• Receipts cannot be created without Purchase Order(s)
• Sale(s) cannot be created without Inventory (and Customers)
• Receivable(s) cannot be created without Sale(s)

I understand your approach but have a question. How are you restoring your database while the application pool (IIS) is running? I regularly restore the database to certain pre-defined states (i.e. pre-sales) when there is a failure but this requires stopping the web server application pool and killing an application process before a restore can be performed.

Are you a Microsoft shop? We use IIS and SQL Server (donning fire suit).

[william_roe]

---

@baxatob wrote:

We were able to make the developers take responsibility for manual testing, code reviews, unit tests and integration tests.
Now the last step is coming that the developers also should do the automated regression testing.

No, the last step should be - to make the developers take responsibility for making the coffee for testers 🙂 

(I'm just kidding. Thank you for your detailed feedback)  

---

The better the automated testing the lazier developers become 

[jsc]

---

@william_roe wrote:

• Product(s) cannot be created with Supplier(s)
• Purchase Order(s) cannot be created without Product(s)
• Receipts cannot be created without Purchase Order(s)
• Sale(s) cannot be created without Inventory (and Customers)
• Receivable(s) cannot be created without Sale(s)

I understand your approach but have a question. How are you restoring your database while the application pool (IIS) is running? I regularly restore the database to certain pre-defined states (i.e. pre-sales) when there is a failure but this requires stopping the web server application pool and killing an application process before a restore can be performed.

Are you a Microsoft shop? We use IIS and SQL Server (donning fire suit).

---

I think we do not have such dependencies, but I would (or at least try to) integrate them in a stable state in our testenvironment.

We use a postgreSQL databases and apacache Tomcat in our web application.

The restoring databases is easy. The service is stopped and started via script as well as the Tomcat service. So we make sure that all services are started properly when login starts.

(in other departments we do similar stuff with MS SQL Server (no problem at all) and Oracle DMBS (bigger and therefore slower to restore)).

No, we are no webshop, we create some sort of ppm-software. (I am not sure if it is allowed to post links to our software here, so I do not...)

[william_roe]

---

@jsc wrote:

---

@william_roe wrote:

• Product(s) cannot be created with Supplier(s)
• Purchase Order(s) cannot be created without Product(s)
• Receipts cannot be created without Purchase Order(s)
• Sale(s) cannot be created without Inventory (and Customers)
• Receivable(s) cannot be created without Sale(s)

I understand your approach but have a question. How are you restoring your database while the application pool (IIS) is running? I regularly restore the database to certain pre-defined states (i.e. pre-sales) when there is a failure but this requires stopping the web server application pool and killing an application process before a restore can be performed.

Are you a Microsoft shop? We use IIS and SQL Server (donning fire suit).

---

.... So we make sure that all services are started properly when login starts.

(in other departments we do similar stuff with MS SQL Server (no problem at all) and Oracle DMBS (bigger and therefore slower to restore)).

---

Sounds as though our next step is to figure out how to start / stop IIS (via script) and how to terminate / launch applications via script.

GREAT DISCUSSIONS!!!

[jsc]

we do it like this (Jscript)

function startTask(taskname, waitTime)
{
  Log.AppendFolder("startTask: " + taskname);
 

  Log.Message("starting task '" + taskname + "'");

  var oShell = new ActiveXObject("WSCript.shell");
  oShell.run('NET START "' + taskname + '"', "1", true);
  Log.Message("trying to start task '" + taskname + "'");
  Delay(waitTime);
  Log.Message("waited for " + waitTime + " ms");

  Log.PopLogFolder();
}

Stopping a task is the same. But we combine it with checking afterwards if the corresponding process does still exist. If the process still exists it is just killed. (script as well)

[chicks]

@william_roe I'm assuming you use 'eval' on consumer side of this method. Can you show what that code looks like? Is the idStr attribute always used? Our templates use ObjectIdentifier

William,  I don't know what you mean by " eval on the consumer side".   idStr is not always used, we use whatever is most specific. sometimes it's only object identifier.  Speed may be a concern, but not to the point where we have spent any time investigating alternattives.

Regards, Curt

[william_roe]

---

@chicks wrote:

@william_roe I'm assuming you use 'eval' on consumer side of this method. Can you show what that code looks like? Is the idStr attribute always used? Our templates use ObjectIdentifier

William,  I don't know what you mean by " eval on the consumer side".

Regards, Curt

---

By consumer side I mean the code which is calling the function. I'm assuming there is code which does a "eval" on the results.