Re: Compare test step

Compare test step

Status:

by on

Purpose

The process of testing often requires to compare test results to expected values. When we need to check a specific value, we can use assertions. However, there are often occasions when we need to compare one documents to another (e.g. XML, JSON). I propose to introduce a new type of test step: Compare test step.

Functionality

Compare test step would allow:

Select documents: users would select source and target document from test step requests, responses, properties and other document holders in SoapUI environment, similarly as the Property Transfer test step does.
Type of comparisson: the user could choose whether the documents have to be identical (both structure and values) or similar (only the structure should be same). Also the user could choose to ignore differences in whitespaces, attributes, or namespaces, or even data types which to ignore (e.g. xsd:dateTime is typically different).
List of items to compare/ignore: the user could list the items which should be ignored, represented by XPath statements. Also there would be an opposite option: a list of XPath statements to check and to ignore the rest of the documents.

Suggestions

The Compare test step can utilize the XMLUnit library for XML and thus expose its function via SoapUI user interface. The Compare step can be similarly used for JSON documents.

Comments

- edited

@TanyaGorbunova,

I believe that this enhancement is applicable for not only Open Source edition, but also for the ReadyAPI.

Most the times, this is the challenge for the end to end automation to compare results from different sources and assert if they matches. Currently, the engineers have to have good programmatic skills to implement and use it.

However, QA / Test engineers might find it difficult those who are not familiar / don't deal with programming in day to day.

Adding this feature may really be helpful for many to achieve the automation.