Forum Discussion

GMSoapUI's avatar
GMSoapUI
Contributor
12 years ago

[Resolved] Test Suite execution is skipping tests

I am using a Groovy Driver Script to execute all tests within my various test suites. This was working in 4.6.2 but after an upgrade to 4.6.3 they are now not running sequentially or not running at all. My Groovy code is as follows:

// for loop for controlling test suite in use
for( vntTestSuite in vntProject.testSuites )
{
vntTestSuite = vntProject.getTestSuiteByName( vntTestSuite.key );
strTestSuiteName = vntTestSuite.name.toString();
if( strTestSuiteName.indexOf( "Groovy" ) == -1 )
{
vntTestSuite.run( null, false );
}
}


I have 56 tests organised in 7 test suites. The last 6 tests in the first three suites are not being executed - they are simply skipped over. The 'upgrade' from 4.6.2 is really causing me problems now - this is the second post about things are now broken and I'm getting tired of bashing my head against a brick wall. There are no errors recorded in either the SoapUI Log or the error log - nothing whatsoever. I have nothing to go off in order to track the problem down. As far as SoapUI Pro is concerned it has executed all the tests, except that it has left 18 tests completely for no apparent reason. Can somebody please help or I'm going to have to start looking for an alternative tool that is more robust between upgrades.

3 Replies

  • I have found the reason for this error. The 'Fail on error' was reset to true in the affected test suites. I had not altered anything in these test suites at all and hadn't ran them for over a week. The first thing I do when creating a test suite is turn 'Fail on error' off as I want everything to run straight through regardless of failures. The affected test suites are ones that were working fine. I upgraded to 4.6.3 and then the flag was reset. >:(
  • Hi,

    thanks for reporting a resolution and glad that you have it working.

    Regarding the "Fail on Error", that property should not be changed with the upgrade for 4.6.3. I have also not seen any reports of similar behavior yet and my own test cases were not affected with the upgrade.

    If you have reproducible steps for this scenario, I can confirm and raise a bug with our development team. Otherwise, again, glad you have the resolution and can keep running your tests!

    Thanks,
    Michael Giller
    SmartBear Software
  • Without rolling back to 4.6.2 and then reinstalling 4.6.3 I cannot get steps to reproduce. It was a simple fix so I'm happy to have this marked as resolved.