Forum Discussion

rancohen's avatar
rancohen
Occasional Contributor
6 years ago

Execute test in random order

Hi,

I have list of tests in test manager and when I pressed the Run tests button, the tests are executes by the order they arranged.
Can I run the tests is random sequence?

Regards,
Ran

5 Replies

  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor

    Short Answer: No.  Executing a test project in TestComplete executes the test items in the order in which they appear in the project.

     

    Long Answer: Maybe.  You will need to, potentially, redesign your test automation project to, instead of running tests as Test items on the project, run them using some sort of script code that has the list of tests to execute and then randomizes the order in code. 

    • rancohen's avatar
      rancohen
      Occasional Contributor
      To simulate user inputs. The user will not run the tests the same sequence as they tested. In some cases there is relation to the order of the tests.

      Ran
      • tristaanogre's avatar
        tristaanogre
        Esteemed Contributor

        rancohen wrote:
        To simulate user inputs. The user will not run the tests the same sequence as they tested. In some cases there is relation to the order of the tests.

        Ran

        That's not something to be tested with randomization of tests... it sounds like you want to test what happens with different "starting conditions".  That's different test cases.  Randomization of test order creates unpredictability in determining what causes any bugs. It would be better to control the tests so you can have better reproduction of what causes bugs.

         

        Regardless, as mentioned before, there is no way "out of the box" to do what you're asking.  You need to write code to randomize your test case execution.  You won't get the nice neat "test items" results that you get now. You will have to find some other method to generate a similar report.

  • shankar_r's avatar
    shankar_r
    Community Hero

    I curious to know why the test needs to be run in random order, I came across the situation where test with random data. 

  • liquidphantom's avatar
    liquidphantom
    Occasional Contributor

    As has been mentioned it's not good practice to randomise the running order of a suite of tests but It's possible if you don't mind getting dirty with some scripting.

     

    Create a script that that adds each test to an array along with a number. You can then pick a random number reads the test from that array and executes that test.

     

    If you only want to call a test once make sure to add a flag column to the array to show that it has been run, and ignore those rows, you could delete the row in the array if it's strait forward in your scripting language of choice which in my case (vbscript) it's not. You'll need to know how to call scripts from another script.

     

    The drawback to this array method though, is that you will not have each case displayed individually in the log tree, but you can get around this by writing the script calls to the log with Log.Message.

     

    Hope this points you in the right direction.