cancel
Showing results for 
Search instead for 
Did you mean: 

Execute test in random order

rancohen
Occasional Contributor

Execute test in random order

Hi,

I have list of tests in test manager and when I pressed the Run tests button, the tests are executes by the order they arranged.
Can I run the tests is random sequence?

Regards,
Ran
5 REPLIES 5
tristaanogre
Community Hero

Re: Execute test in random order

Short Answer: No.  Executing a test project in TestComplete executes the test items in the order in which they appear in the project.

 

Long Answer: Maybe.  You will need to, potentially, redesign your test automation project to, instead of running tests as Test items on the project, run them using some sort of script code that has the list of tests to execute and then randomizes the order in code. 


Robert Martin
[Hall of Fame]
Please consider giving a Kudo if I write good stuff
----

Why automate?  I do automated testing because there's only so much a human being can do and remain healthy.  Sleep is a requirement.  So, while people sleep, automation that I create does what I've described above in order to make sure that nothing gets past the final defense of the testing group.
I love good food, good books, good friends, and good fun.

Mysterious Gremlin Master
Vegas Thrill Rider
Extensions available
shankar_r
Community Hero

Re: Execute test in random order

I curious to know why the test needs to be run in random order, I came across the situation where test with random data. 


Thanks
Shankar R

LinkedIn | CG-VAK Software | Bitbucket | shankarr.75@gmail.com

“You must expect great things from you, before you can do them”

Extension Available

rancohen
Occasional Contributor

Re: Execute test in random order

To simulate user inputs. The user will not run the tests the same sequence as they tested. In some cases there is relation to the order of the tests.

Ran
tristaanogre
Community Hero

Re: Execute test in random order


@rancohen wrote:
To simulate user inputs. The user will not run the tests the same sequence as they tested. In some cases there is relation to the order of the tests.

Ran

That's not something to be tested with randomization of tests... it sounds like you want to test what happens with different "starting conditions".  That's different test cases.  Randomization of test order creates unpredictability in determining what causes any bugs. It would be better to control the tests so you can have better reproduction of what causes bugs.

 

Regardless, as mentioned before, there is no way "out of the box" to do what you're asking.  You need to write code to randomize your test case execution.  You won't get the nice neat "test items" results that you get now. You will have to find some other method to generate a similar report.


Robert Martin
[Hall of Fame]
Please consider giving a Kudo if I write good stuff
----

Why automate?  I do automated testing because there's only so much a human being can do and remain healthy.  Sleep is a requirement.  So, while people sleep, automation that I create does what I've described above in order to make sure that nothing gets past the final defense of the testing group.
I love good food, good books, good friends, and good fun.

Mysterious Gremlin Master
Vegas Thrill Rider
Extensions available
liquidphantom
Occasional Contributor

Re: Execute test in random order

As has been mentioned it's not good practice to randomise the running order of a suite of tests but It's possible if you don't mind getting dirty with some scripting.

 

Create a script that that adds each test to an array along with a number. You can then pick a random number reads the test from that array and executes that test.

 

If you only want to call a test once make sure to add a flag column to the array to show that it has been run, and ignore those rows, you could delete the row in the array if it's strait forward in your scripting language of choice which in my case (vbscript) it's not. You'll need to know how to call scripts from another script.

 

The drawback to this array method though, is that you will not have each case displayed individually in the log tree, but you can get around this by writing the script calls to the log with Log.Message.

 

Hope this points you in the right direction.

New Here?
Join us and watch the welcome video:
Announcements
Top Kudoed Authors