Questions concerning post run values

Hello Craig,

That's a whole lot of questions you got there!
I'll do my best to answer them all:
One thing that looks common for all the question is the separation of global test execution data and individual component data, so let's clear that out.

Component data:
requests, running, completed, queued, discarded, failed.
These metrics are local to all individual component, what counts as a request / failure and discarded is usually local to that individual component.
SoapUI runner for example treats a failed test-step as a failure, where the HTTP runner treats a request that never returns as a failure.

Project execution data:
requests, failed requests, assertions, failed assertions.
These metrics are global to the project test execution and by assertions here means the limits on metrics of a components average response time, currently running, transactions per second and so on.

What does the 934 Failures represent?
Project-level failed assertions.

What sort of failures are they - loadUI assertion failures or soapUI assertion failures?
LoadUI project level assertions. SoapUI as a component can choose whether to treat failed test-steps as project Failures or not.

Also if 321 Failed - does that mean 321 Failed assertions?
Not necessarily, since this is metrics from the component. Check your SoapUI runners configuration.

If so, why doesn't that value match the the Assertion values on the report?
This depends, remember that you can configure failed test-steps to generate failures, that way you could end up with more project-level failures than component failures.

If it does not represent Assertion Failures but request failures - shouldn't it match the 934 value?
As i explained in the question above, not necessarily. But there is a known issue related to reporting sometimes not being consistent.
This issue is referred to as LOADUI-1486.

what does the “Total number of assertions” represent?
The project level assertions, the number of times a metric-assertion has failed.

What exactly does Discarded mean?
Discarded means that a request is thrown away because a request-queue has grown too large, the default limit is 1000 requests.

How are the Discarded requests being taken into account overall?
They are not treated as failures and as such does not appear on the report.

Why don't any of these numbers add up?
Possibly because of the bug mentioned above.

Are some of the Failures that are showing up, Failures of the Assertions that exist on the soapUI Project rather than the loadUI assertions?
Possibly, the best way to find out is to check the settings of the SoapUI component.


I hope that answers all your questions!
Do get back on this topic should you have further questions.

Best,
Mikael

Thank you for your reply to my questions. We'll be discussing your answers at our weekly performance meeting and how to incorporate them into our different testing approaches.