Script assertion passes individually but fails in running testSuite

[casp8]

Hi all, 

I have a script assertion to assert whenether a response is null:

assert messageExchange.hasResponse() == false

When I run the assertion or request individually, it passes.

However when I run the entire testsuite, the suite fails (while response is still empty). The step doesn't turn red but stays grey.

How can I resolve this? I'm at ReadyAPI version 2.6.0.

[richie]

Hey,

If the assertion failed, i'd suggest sticking a delay stop in before the step with assertion, but if it's not failing it sounds like something else is going on.

What does your logging say?

Cheers,

Richie

[casp8]

Thanks for your suggestion! I put a delay in the testcase before the step, and tried a delay in the script assertion but the problems persists. 

Troubleshooting I now noticed that this error is also thrown when there is no assertion. So the actual problem is that this step is never completed when ran in a testcase, but runs properly when fired as an individual request.

The test is to see whenether a request without a certificate gets a null response on an secure port. This apparantly throws an javax.net.ssl.SSLHandshakeException.

My http log shows:

• [write] I/O error: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

The error log shows:

• Mon May 27 10:01:56 CEST 2019: ERROR: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Which is not unexpected given the testsituation, but how to make soapui run past this Exception?

[casp8]

Thanks for your suggestion! I've tried a delay but the problem persists. 

Troubleshooting I realized it had nothing to do with the script assertion but with the request. Without an assertion the step is also never completed, and stays 'grey. 

The teststep fires a request without a keystore on a secure port to check that whenether the response is null. This runs a single request, but does not in a testcase. 

Apparantly this is because this attempt throws an exception; (@error log)

• Mon May 27 10:35:19 CEST 2019: ERROR: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

And this causes ReadyAPI to stop the testplan. When ran as a single request, the exception is also thrown, but doesn't halt execution, and the scipt assertion is tested normally. 

[richie]

Hi,

A test step will execute but remain 'grey' if there are either no assertions on the step or if the step wasnt executed within the test case.

Im a bit unsure what youre after, but if you're trying to ensure all the test steps within the test case executes despite if a failure occurs you can alter this behaviour by opening up thr test case options form. By default there is a checkbox, something like "stop execution of test if any step fails" that can be unchecked. (Sorry its 2am in the morning and im typing this on my phone so i cant give you the exact checkbox name)
Once unchecked, this will enable the test case to execute all the way through, despite any steps in the test failing.

Does that help?

Ta

Richie

[Olga_T]

Hi,

Thanks for looking into this, @richie !
@casp8, has the latest suggestion worked for you?

[TanyaYatskovska]

Hi @casp8,

My two cents, I suggest that you download and install the latest maintenance build. Perhaps, it will handle the exception a bit differently. You can download the installation file here:

https://support.smartbear.com/downloads/readyapi/maintenance