SoapUI Reporting of concise JsonPath Match failures
We are currently trying to implement SoapUI into the project to run regression tests and seem to be struggling to create useful reports when our assertions - which are mostly based on json path assertions - fail.
Our setup is as follows: we call a few services and then assert a response from a DB or from the service itself by using an assertion step containing numerous JsonPath assertions.
The reports of errors are currently in this format: (All project related words are replaced by <>)
Time Taken: 0
Timestamp: Tue Jul 16 14:31:36 CEST 2019
----------------- Messages ------------------------------
[match <parameter1>] JsonPath Match FAIL
[match <parameter3>] JsonPath Match FAIL
[match <parameter6>] JsonPath Match FAIL
We would like to know what exactly went wrong instead of having to look at the executed tests, so we need to have the expected value and the actual value in the report, just like we can look at them when we directly execute the assertion, which looks like this:
Comparison failed. Path: [<path>]; Expected value: [<expectedValue>]; Actual value: [<actualValue>]
How would we add the messages we receive by the singular assertions to the complete log?
Thanks in advance for the help
Solved! Go to Solution.
If you see current sample messages in the above, all the have the same name and difficult to distinguish because, all the assertions have the same name.
I am required by my employer to blank the assertion names, I can assure you that they are more eligable and understandable. We could even go further and put the expected value into the name, but that will still not make it possible to see what value was actually returned in the report, which is the core problem.
In case this shows up in anyones searches: The problem was related to an internal issue with SoapUI.
When using assertion-steps, the jsonpath-errors are not properly passed to the reports, using assertions appended to the REST-step itself is a possible solution.
The problem with the assertion-steps is hopefully going to be solved in one of the next releases.