Forum Discussion

Mason's avatar
Mason
Frequent Contributor
15 years ago

Running Tests from Project level (parallel or sequential)?

Can someone clarify for me if the Test Suites run sequentially or in parallel when run from the Project level?  I thought it was supposed to be sequential but I'm seeing behavior that suggests it is parallel.

Thanks,
Mason

9 Replies

  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    SmartBear Alumni (Retired)
    Hi,

    by default it should be sequential, but you can change to parallel in the toolbar. What is indicating parallel execution?

    regards!

    /Ole
    eviware.com
  • Mason's avatar
    Mason
    Frequent Contributor
    eviware support wrote:

    Hi,

    by default it should be sequential, but you can change to parallel in the toolbar. What is indicating parallel execution?

    regards!

    /Ole
    eviware.com


    Which toolbar...at the project level?  I see no toolbar button/option on the Project pages to run TestSuites in parallel or sequential.  I see this at the Suite level for Test Cases.  Just to be clear, I'm talking about the sequence for Test Suites, not Test Cases.
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    SmartBear Alumni (Retired)
    Hi!

    sorry.. maybe I misunderstood; in 3.0-beta there is a TestSuites tab in the Project window much like the TestCases in the TestSuite window.. but maybe you are running from the command-line in 2.5.2? Or via the Project Coverage tab?

    regards,

    /Ole
    eviware.com
  • Mason's avatar
    Mason
    Frequent Contributor
    We are running from the UI.  We need to know if the Test Suites run in parallel or are they in sequence in the order they appear (top to bottom) in the UI.
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    SmartBear Alumni (Retired)
    Hi!

    hmm.. how are you running them in the UI if you are not using the 3.0 beta? Via the "Launch TestRunner" menu option? (or are you using the beta?)

    regards!

    /Ole
    eviware.com
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    SmartBear Alumni (Retired)
    Hi!

    ah ok.. this should still run them sequentially.. what indications are you getting of the opposite?

    regards!

    /Ole
    eviware.com
  • Mason's avatar
    Mason
    Frequent Contributor
    Watching the coverage UI during a test run strongly suggests they are not sequential as can be seen in the linked screencast. 

    http://screencast.com/t/W1idHEXTCK

    Note the three suites towards the bottom of the coverage UI.  The first suite shows activity, then seconds later, the other two appear to be running.  There is a long pause then towards the end (after 20 seconds or so), all three suites indicate activity and the test run completes.

    Furthermore, and this would be a much larger issue, our setup scripts at the TestSuite level are not running when running all TestSuites from the Project level. 

    We were able to prove this as our groovy script writes out property values during it's execution and none of the properties were being updated.  We then cleared out all of the properties, ran the project and the properties were blank at the end of the test run.

    We're hoping this can be corrected in a nightly build.
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    SmartBear Alumni (Retired)
    Hi Mason,

    I urge you to try the 3.0 beta instead for this, the project runner has been overhauled for many of the reasons you are pointing out. A fixed 2.5.X will be available, but it will be after the 3.0 release (no date has been set internally).

    Sorry for the inconvenience.

    regards!

    /Ole
    eviware.com