Forum Discussion

sanjeevkumar7's avatar
sanjeevkumar7
Occasional Contributor
14 years ago

Best approach for version compatibitliy testing

Hi- I would like to know any suggestions/comments about the best way to perform version compatibility testing for the below scenario.

I want to pass in the same input for different versions of the same webservice, to make sure if doesn't break any old functionality or returning a wrong value. Expected value is captured using Run 1.

Run 1 : Web service version 1

Step 1: Get input values from datasource
Step 2: Send a request and validate the response
Step 3: Groovy script. Count number of nodes return. Insert the response values for each node to a sql database. Each response produces "n" number of rows and "k" columns
Step 4: DataLoop - has "m" rows

At the end of the run1, my sql table has "n X m" rows containing "k" columns each.

Run 2: web service version 2

I would like to use the same input values from the datasource and perform Step 1 and Step2.

Now my question is
1) Do I create a subreport instead of step 2 and compare the response against the db values captured from run1?
2) Do I write it to another sql table and then create another step to validate one table against another?

If choice 1, I would have to compare "k" columns for "n X m" number of rows. Is it possible to perform that using groovy script?
I have no idea how to deal with solution 2.

Appreciate if someone had dealt with the same kind of scenario before and would like to give me some suggestion.

Thanks!
Sanjeev
  • Hi Sanjeev!

    Can't you send requests to both versions in two subsequent TestSteps and comparing them on the fly?

    sanjeevkumar7 wrote:
    1) Do I create a subreport instead of step 2 and compare the response against the db values captured from run1?

    A subreport? Creating a report instead of sending a request and validate the response? Sorry, I don't follow you.

    sanjeevkumar7 wrote:
    2) Do I write it to another sql table and then create another step to validate one table against another?

    I'd go for that. You could also add an extra column containing the Web Service version.

    Regards!

    /Henrik
    eviware.com
  • sanjeevkumar7's avatar
    sanjeevkumar7
    Occasional Contributor
    Hi Henrik


    Can't you send requests to both versions in two subsequent TestSteps and comparing them on the fly?


    That's great. I never thought that way. I still had to store the response in the database. So I chose the 2nd method.

    Added two columns - run_id and version. run_id increments for each run.

    I send the same request for two different versions, capture the required response, use a stored procedure to compare the rows of two versions with the same run_id and write the result to another table.

    May be all of the above could have been done with groovy, but thought of using sql to do the heavy lifting.

    Is there a way to read the results table (which will contain the failed cases ) and populate in the report?

    Your tip helped me move forward with this. Thanks.

    -Sanjeev