Forum Discussion

pb1's avatar
pb1
Contributor
6 years ago

SoapUI Reporting of concise JsonPath Match failures

Hello

 

We are currently trying to implement SoapUI into the project to run regression tests and seem to be struggling to create useful reports when our assertions - which are mostly based on json path assertions - fail. 

 

Our setup is as follows: we call a few services and then assert a response from a DB or from the service itself by using an assertion step containing numerous JsonPath assertions. 

 

The reports of errors are currently in this format: (All project related words are replaced by <>)

Status: FAIL
Time Taken: 0
Size: 0
Timestamp: Tue Jul 16 14:31:36 CEST 2019
TestStep: <stepName>
----------------- Messages ------------------------------
[match <parameter1>] JsonPath Match FAIL
[match <parameter3>] JsonPath Match FAIL
[match <parameter6>] JsonPath Match FAIL

We would like to know what exactly went wrong instead of having to look at the executed tests, so we need to have the expected value and the actual value in the report, just like we can look at them when we directly execute the assertion, which looks like this:

Comparison failed. Path: [<path>]; Expected value: [<expectedValue>]; Actual value: [<actualValue>]

How would we add the messages we receive by the singular assertions to the complete log?

 

Thanks in advance for the help

pb1

Assertions
JsonPath
Reporting
REST
  • pb1's avatar
    pb1
    6 years ago

    In case this shows up in anyones searches: The problem was related to an internal issue with SoapUI.

     

    When using assertion-steps, the jsonpath-errors are not properly passed to the reports, using assertions appended to the REST-step itself is a possible solution.

     

    The problem with the assertion-steps is hopefully going to be solved in one of the next releases.

  • nmrao's avatar
    nmrao
    Champion Level 3
    6 years ago
    First it requires to name the assertion appropriately so that it can be easily understand based on the name.

    If you see current sample messages in the above, all the have the same name and difficult to distinguish because, all the assertions have the same name.
    • pb1's avatar
      pb1
      Contributor
      6 years ago

      Hi Rao

       

      I am required by my employer to blank the assertion names, I can assure you that they are more eligable and understandable. We could even go further and put the expected value into the name, but that will still not make it possible to see what value was actually returned in the report, which is the core problem.

      • pb1's avatar
        pb1
        Contributor
        6 years ago

        In case this shows up in anyones searches: The problem was related to an internal issue with SoapUI.

         

        When using assertion-steps, the jsonpath-errors are not properly passed to the reports, using assertions appended to the REST-step itself is a possible solution.

         

        The problem with the assertion-steps is hopefully going to be solved in one of the next releases.