Testrunner doesn't export result for failed JDBC DataSource

[hisaak]

I use -a -A -I options with testrunner.sh to run a test suite with multiple test cases. These options should advise testrunner.sh to write results for all test steps into the separate files. However, when DataSource step of JDBC type fails somewhere in the middle of a test suite, it doesn't write any file for the whole test case. Such a test case is correctly marked as failed in the final short summary (-r) and also test suite report contains appropriate error message. So the only one problem lies in missing test case results.

This is a serious issue for me since I detect failure (within Jenkins job) for whole test suite by looking for files ending with "FAILED.txt".

If someone wants to replicate the issue, it just takes to set wrong credentials for database connection. In my case, I get the following error from Oracle database: java.sql.SQLException: ORA-01017: invalid username/password; logon denied.

Any help is appreciated.

soapUI Pro 4.0.1 on Fedora Linux x86_64

[SmartBear_Suppo]

Hello,

thank you for letting us know! I've opened issue for tracking this (id SOAPUI-2977).

Regards,
Vladimir
SmartBear Software

[hisaak]

Is there any time frame estimate for this issue to be resolved? I'd like to adjust our plans regading the use of JDBC DataSources.

[SmartBear_Suppo]

Hi!

hmm.. I'm looking into this - do you get any errors in the console output of the test when this teststep executes?

regards!

/Ole
SmartBear Software

[hisaak]

do you get any errors in the console output of the test when this teststep executes?

Yes, i do. The error output in the test console is just fine as you can see in attached output.txt. I got this file from this command:

$ /opt/soapui-pro-4.0.1/bin/testrunner.sh -F HTML -R 'TestSuite Report' -Pdatabase=my_oracle_database -Penvironment=my_environment -I -a -s 'TestSuite - CheckLogin' -f output -r CheckLoginWS-soapui-project.xml > output.txt 2>&1

$ ls output
TestSuite--CheckLogin
TestSuite__CheckLogin-TestCase__Existing_userName_file-checkUserName__existing_userName-0-OK.txt
TestSuite__CheckLogin-TestCase__Existing_userName_file-DataSource__CUID_userName_file-0-OK.txt
TestSuite__CheckLogin-TestCase__Incorrect_userName-checkUserName__spaces-0-OK.txt
TestSuite__CheckLogin-TestCase__Incorrect_userName-DataSource__CUID-0-OK.txt
TestSuite__CheckLogin-TestCase__New_userName-checkUserName__new_userName-0-OK.txt
TestSuite__CheckLogin-TestCase__New_userName-DataSource__CUID_file-0-OK.txt

As you can see there are only "OK" files under output directory. I also checked soapui.log and soapui-errors.log files and there is nothing interesting at first glance. Anyway, if you want I can attach them as well.

Hope that helps ...

[SmartBear_Suppo]

Hi!

ok, thanks for this - the problem is that the runner outputs files for failed TestSteps but your TestCase fails before running any steps at all (it fails during initialization) and since there are no files created for the TestCase itself you won't get the marker file you are looking for. One possible solution might be to add a script in the TearDown of your TestCase that checks if it failed and creates the file manually!?

Does that make any sense?

regards!

/Ole
SmartBear Software

[hisaak]

SmartBear Support wrote:
Hi! ok, thanks for this - the problem is that the runner outputs files for failed TestSteps but your TestCase fails before running any steps at all (it fails during initialization)

No, that's not the case. JDBC connect is within the first step of the TestCase so I expect it to throw some FAILED output file when it fails. Just compare it to File based Data Source. When I replace JDBC step by File data source step and the file is not found, then I get FAILED file in output for this TestCase step. And that's what I want for database based data source as well.

[SmartBear_Suppo]

Hi,

unfortunately these two work differently internally - the JDBC DataSource will fail during initialization of the TestCase, the File DataSource will fail first when the TestStep is executed - I've reported this as a bug internally but for now maybe you could try my suggested solution!?

regards!

/Ole
SmartBear Software

[hisaak]

Ok, I understand your point. As a workaround, your suggestion might work and the implementation is quite easy. However, I plan to train some "not so skilled" testers to work with soapUI on their own so they can create simple TestCases without any coding. And I think it wouldn't be much intuitive for them to use special TearDown script for TestCases including JDBC and not use any script for the rest of TestCases.

This is the main reason why I'd like to use JDBC data sources just like any other data sources we use (CSV, Excel). Let's call it convenience or uniformity.

Regards,
hisaak

[SmartBear_Suppo]

Hi,

definitely they should all work the same - we'll get back to you here in the forum when we've fixed this - I'm sorry for the inconvenience for now...

regards!

/Ole
SmartBear Software