Forum Discussion

nishanj's avatar
nishanj
Occasional Contributor
12 years ago

[Reg]Inconsistent use of CDATA between functional & loadtest

Much of our useful data is embedded in CDATA fields in our Response messages.
For example: <response><![CDATA[<status>0</status>]]></response>

We frequently use “Contains” assertions to check for expected values:
For example: “<status>0</status>”
Which works because the assertion finds the quoted string.

However, when the same TestCase is run via a Load Test, the previously passing assertions fail because the string passed to the Assertion by SoapUI would appear to be:
<response>&lt;status&gt;0&lt;/status&gt;</response>

Initially we adopted the following approach. We updated the Contains Assertion to “Use token as Regular Expression” and accepted either string variant.
(?s).*((<status>0</status>)|(&lt;status&gt;0&lt;/status&gt)).*

However, with increasingly complex regular expressions and an increasing number of assertions to maintain we cannot continue with this approach.

1. My assertion is that SoapUI should by default provide a consistent interface for assertion processing regardless of whether the test step is being used in the normal or the load environment.
2. Is there a configuration option to enforce such consistency?
3. If the above does not exist, what do you suggest as the best approach for new test assertions and for revisiting existing test projects? I am aware of the CDATA removal Event Handler option that is documented on the soapui.org website.

5 Replies

  • Hi,

    1. My assertion is that SoapUI should by default provide a consistent interface for assertion processing regardless of whether the test step is being used in the normal or the load environment.


    I agree, I'm not sure why the result would encode the tags as &lt; and &gt; in a load test.

    2. Is there a configuration option to enforce such consistency?


    No, not for this scenario. There is something similar for endpoints at File -> Preferences -> HTTP Settings, Pre-encoded Endpoints.

    3. If the above does not exist, what do you suggest as the best approach for new test assertions and for revisiting existing test projects? I am aware of the CDATA removal Event Handler option that is documented on the soapui.org website.


    The best approach may be to capture the actual value and compare via a groovy assertion.
    Using XMLHolder can do this:
    http://testautomationnoob.blogspot.com/ ... older.html

    Regards,
    Marcus
    SmartBear Support
  • Hi,

    I wasn't able to reproduce this doing a load test. The tags in the CDATA did not change. Can you give more information on the type of load test you are doing and how many threads you are using?




    Regards,
    Marcus
    SmartBear Support
  • nishanj's avatar
    nishanj
    Occasional Contributor
    Hi Marcus,

    Thanks for checking this.

    Under normal SoapUI operation we receive XML in CDATA sections. Partial real example:
    ... <status>WARNING</status><type>xml</type><processingStatusCode>1103</processingStatusCode><value></value></return>]]></SubmitTransactionResult>

    Whereas running the same Test Case as a Load Test in SoapUI (including V4.6.3) even with 1 thread, simple strategy and 1 test run, the "contains" assertion has to also accept:
    ... &lt;status&gt;WARNING&lt;/status&gt;&lt;type&gt;xml&lt;/type&gt;&lt;processingStatusCode&gt;1103&lt;/processingStatusCode&gt;&lt;value&gt;&lt;/value&gt;&lt;/return&gt;</SubmitTransactionResult>

    Regards
  • I agree that it'd be nice to achieve consistency between functional tests and load tests.

    Load testing is not a priority feature in SoapUI – our recommendation is to use our free tool LoadUI, which can import SoapUI projects, for serious load tests – but I'll add this issue to the backlog for prioritization.

    Regards,
    Manne
    SoapUI developer
  • nishanj's avatar
    nishanj
    Occasional Contributor
    Please do add this issue to the backlog and if other clients indicate this as a problem for them as well, take this into account when prioritising a resolution. This problem will affect how we use SoapUI and SoapUI-Pro.

    Thank you for your responses.