Forum Discussion

Dan1Dan's avatar
Dan1Dan
New Contributor
12 years ago

Number of Tests

We have a service that returns a huge order object/xml. Does anyone know of any documented strategies or methodologies for narrowing down the # of tests necessary? Is it normal to test every element (1, few, many) and the valid combinations in a WSDL?

Note: I am familiar with test techniques such as decision tables, state diagrams, domain testing, … , but we typically aren’t dealing with hundreds (or thousands) of fields.

5 Replies

  • nmrao's avatar
    nmrao
    Community Hero
    Understand your perspective. I see this is subjective, depends on various reasons and test coverage should be atleast, and add tests to increase possibility to break the service/application rather than based on just number of tests.
    1. verified uses cases that particular service/operations supports and they work
    2. tests must cover as per the documentation of service, response codes etc
    3. how the application works if any of the mandatory elements missing, i mean identify certain pattern and test those. It is not much fruitful results applying the same pattern to each of element, in the end probably get same kind of response. Of course, that does mean optional elements can be ignored. See if the default value mentioned for the optional elements working as expected etc
    4. Check if there are any enumerations that any elements follows/uses, try to break there if allows other than intended values for that specific elements.
    5. If there are some choice elements in the request, cover possible both options.
    6. Add boundary conditions

    Some of my inputs i could think of, I am sure you can come up some more. Hoping that understanding the wsdl/underlying xml schema would help you well to draw the tests.
  • Dan1Dan's avatar
    Dan1Dan
    New Contributor
    Thanks. I have team members using SoapUI, but I haven’t personally, so please excuse my question if the answer is easily found. If I have a template XML with 100’s of fields, is there an easy way to:

    1) Dump it into a format where I can add data in bulk (like adding excel rows)?

    2) Turn the Excel rows back into individual XMLs?

    Note: My team members are mapping fields from a datasource to a template request, but I need to do it at a much faster pace ...
  • PaulDonny's avatar
    PaulDonny
    Regular Contributor
    Dan1Dan,

    Sounds like you need to set up data driven testing. The easiest way to achieve this will be with groovy scripts and a data source. You can have the groovy scripts manipulate the XML within the request and implant the data as needed. If there is multiple different services being called I find it easiest to use properties and then put the property expansion within the call itself. As the last step, have it verify that the test case passed, log the results of the test run (I typically use a test name in my data source) with a pass or fail and if a fail, log the step and the assertion that failed then have it loop back to the beginning of the steps, set up using the next row from the data source and run that one.
  • Dan1Dan's avatar
    Dan1Dan
    New Contributor
    PaulM - I am trying to avoid mapping each field one by one, since there are hundreds and they still haven't locked down the schema. Is there a way to do what you are saying in bulk?

    1) Excel lets you import an XML, so I can get the fields/columns in excel

    2) I can add as much data (request rows) as I want

    3) But, I don't yet have a good way to turn the rows back to XMLs ...