I've found another problem with results from soapUI Runner.
The result Status of failed call to Request1 TestCase (this is the Run TestCase TestStep) is wrong.
Call finished with status 500, but with no assertions on response (I don't need functional assertions when I test performance), the Status field returned has value 'true', which is far from real result. Here's a proof:
10:02:18,971 DEBUG [HttpClientSupport$SoapUIHttpClient] Stale connection check
10:02:18,972 DEBUG [HttpClientSupport$SoapUIHttpClient] Attempt 1 to execute request
10:02:18,972 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Sending request: POST /ws/IndvPrsn HTTP/1.1
10:02:19,004 DEBUG [SoapUIMultiThreadedHttpConnectionManager$SoapUIDefaultClientConnection] Receiving response: HTTP/1.1 500 Internal Server Error
10:02:19,004 DEBUG [HttpClientSupport$SoapUIHttpClient] Connection can be kept alive indefinitely
message==[TimeTaken:38, Timestamp:1342425738964, Status:true, ID:Request1]
As You can see, request finished with HTTP Status Code = 500 (Internal Server Error), but Status in the message says the request has finished successfully (Status = 'true').
With acces to soapui_context (I have it on the TestCase level, but not on Request TestStep or Run TestCase TestStep level) I can check manually the real Status code in custom component, eg.:
soapui_context = message.get('soapui_context')
status_code = soapui_context.get('httpResponse').getStatusCode()
if ( status_code != 200 ) {
// Here we know that request finished with error
}
Another thing is value in TimeTaken field. The returned value is 38, while "Sending request" 10:02:19,004 - 10:02:18,972 "Receiving response" = 32.
Here the difference is only 6 ms, but I've had earlier results, where TimeTaken was greater than 700 ms, but real execution time was below 100ms.
It sometimes happens with the first TriggerTimestamp from VU Generator (eg. Fixed Rate Component), when soapUIComponent has some delays while initializing.
As far as I know, the reason of such big difference in TimeTaken value is usage of TriggerTimestamp value as a start, instead of timestamp from "Sending request".
Again with access to soapui_context I can extract better value for TimeTaken:
TimeTaken = soapui_context.get('httpResponse').getTimeTaken()
But as before, the soapui_context is only available on the TestCase level, but now below that.
Regards,
Tomek
