SoapUI Reporting of concise JsonPath Match failures
We are currently trying to implement SoapUI into the project to run regression tests and seem to be struggling to create useful reports when our assertions - which are mostly based on json path assertions - fail.
Our setup is as follows: we call a few services and then assert a response from a DB or from the service itself by using an assertion step containing numerous JsonPath assertions.
The reports of errors are currently in this format: (All project related words are replaced by <>)
Status: FAIL Time Taken: 0 Size: 0 Timestamp: Tue Jul 16 14:31:36 CEST 2019 TestStep: <stepName> ----------------- Messages ------------------------------ [match <parameter1>] JsonPath Match FAIL [match <parameter3>] JsonPath Match FAIL [match <parameter6>] JsonPath Match FAIL
We would like to know what exactly went wrong instead of having to look at the executed tests, so we need to have the expected value and the actual value in the report, just like we can look at them when we directly execute the assertion, which looks like this:
Comparison failed. Path: [<path>]; Expected value: [<expectedValue>]; Actual value: [<actualValue>]
How would we add the messages we receive by the singular assertions to the complete log?
Re: SoapUI Reporting of concise JsonPath Match failures
I am required by my employer to blank the assertion names, I can assure you that they are more eligable and understandable. We could even go further and put the expected value into the name, but that will still not make it possible to see what value was actually returned in the report, which is the core problem.