cancel
Showing results for 
Search instead for 
Did you mean: 

Script assertion passes individually but fails in running testSuite

Occasional Contributor

Script assertion passes individually but fails in running testSuite

Hi all, 

 

I have a script assertion to assert whenether a response is null:

assert messageExchange.hasResponse() == false

When I run the assertion or request individually, it passes.

However when I run the entire testsuite, the suite fails (while response is still empty). The step doesn't turn red but stays grey.

How can I resolve this? I'm at ReadyAPI version 2.6.0.

6 REPLIES 6
Community Hero

Re: Script assertion passes individually but fails in running testSuite

Hey,

If the assertion failed, i'd suggest sticking a delay stop in before the step with assertion, but if it's not failing it sounds like something else is going on.

What does your logging say?

Cheers,

Richie
if this helped answer the post, could you please mark it as 'solved'? Also if you consider whether the title of your post is relevant? Perhaps if the post is solved, it might make sense to update the title of the post to something more descriptive? This will help people when searching for problems.
Occasional Contributor

Re: Script assertion passes individually but fails in running testSuite

Thanks for your suggestion! I put a delay in the testcase before the step, and tried a delay in the script assertion but the problems persists. 

 

Troubleshooting I now noticed that this error is also thrown when there is no assertion. So the actual problem is that this step is never completed when ran in a testcase, but runs properly when fired as an individual request.

The test is to see whenether a request without a certificate gets a null response on an secure port. This apparantly throws an javax.net.ssl.SSLHandshakeException.

 

My http log shows:

  • [write] I/O error: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

The error log shows:

  • Mon May 27 10:01:56 CEST 2019: ERROR: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Which is not unexpected given the testsituation, but how to make soapui run past this Exception?

 

Occasional Contributor

Re: Script assertion passes individually but fails in running testSuite

Thanks for your suggestion! I've tried a delay but the problem persists. 

 

Troubleshooting I realized it had nothing to do with the script assertion but with the request. Without an assertion the step is also never completed, and stays 'grey. 

The teststep fires a request without a keystore on a secure port to check that whenether the response is null. This runs a single request, but does not in a testcase. 

 

Apparantly this is because this attempt throws an exception; (@error log)

  • Mon May 27 10:35:19 CEST 2019: ERROR: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

And this causes ReadyAPI to stop the testplan. When ran as a single request, the exception is also thrown, but doesn't halt execution, and the scipt assertion is tested normally. 

Community Hero

Re: Script assertion passes individually but fails in running testSuite

Hi,

A test step will execute but remain 'grey' if there are either no assertions on the step or if the step wasnt executed within the test case.

Im a bit unsure what youre after, but if you're trying to ensure all the test steps within the test case executes despite if a failure occurs you can alter this behaviour by opening up thr test case options form. By default there is a checkbox, something like "stop execution of test if any step fails" that can be unchecked. (Sorry its 2am in the morning and im typing this on my phone so i cant give you the exact checkbox name)
Once unchecked, this will enable the test case to execute all the way through, despite any steps in the test failing.

Does that help?

Ta

Richie
if this helped answer the post, could you please mark it as 'solved'? Also if you consider whether the title of your post is relevant? Perhaps if the post is solved, it might make sense to update the title of the post to something more descriptive? This will help people when searching for problems.
Moderator

Re: Script assertion passes individually but fails in running testSuite

Hi,

 

Thanks for looking into this, @richie !
@casp8, has the latest suggestion worked for you?

 


Olga Terentieva
SmartBear Assistant Community Manager

Community Manager

Re: Script assertion passes individually but fails in running testSuite

Hi @casp8,

My two cents, I suggest that you download and install the latest maintenance build. Perhaps, it will handle the exception a bit differently. You can download the installation file here:

https://support.smartbear.com/downloads/readyapi/maintenance

---------
Tanya Gorbunova
SmartBear Community Manager

Did my reply answer your question? Give Kudos or Accept it as a Solution to help others.↓↓↓↓↓
New Here?
Join us and watch the welcome video:
Top Kudoed Authors