Forum Discussion

adamali's avatar
adamali
Occasional Contributor
7 years ago

automated testing - drive all test parameters (including http verb) from excel sheet

Hello,

 

I'm trying to standup automated testing capability for APIs using ReadyAPI (soapUI, etc.). for the sake of speed, I need to be able to drive all request (REST request) parameters from excel sheet including the http verb. I did manage to get everything as a parameter from the excel sheet even assertions but I couldn't drive the http verb/method (POST, GET, PUT, & DELETE) from the excelsheet.

 

My ultimate goal is to have a 4 step generic test case 

 

dataSourceLogin (Excel login)

   loginRequest

dataSourceLoopLogin

 

dataSourceScenarios (Excel)

   request (POST, GET, PUT or DELETE)

dataSourceScenarios Loop

 

As per the above, the test QA analyst after developing the test scenarios, all he/she needs to do is to just fill in a row for every scenario or multiple row for a complex scenario.

 

After a chat with one of SmartBear engineers, he confirmed that it is impossible to do that as this is how it was designed. Now I'm in a very difficult situation as I'm half way through and I'm stuck. He said that different requests with different verbs have different composition. 

 

I was thinking to do the below workaround but I need some help:

 

What if I define a generic loop that has 5 steps, 1 dataSource, 4 steps (REST Requests) and 1 dataSource Loop
 
testScnariosDataSource
  PostRequest
  GetRequest
  PutRequest
  DeleteRequest
testScenarioDataLoop
 
The requests would all get their parameters (headers, and body if any from thetestScnariosDataSource)
 
then in the testScnariosDataSource will have a column for http verb/method (POST, GET, PUT, Delete). at the runtime if the verb/method (plus "Request")  in the datasource doesn't match the Step name, then I will skip it.
 
The question is how can apply the above logic using Events and how can I skip a request step.
 
Would appreciate any help with the above or even another idea to achieve the same.
 
Another way I'm thinking about it which might do it but not so elegant, is to have 4 rows in the excel-sheet that correspond to the 4 requests and in the excel sheet, I fill one row with the active scenario I want to test and the rest of the 3 with dummy calls to a service (google.com or something) that always return success.
 
 Excel DataSource:
     POST  Active test case
     GET  Dummy
     PUT  Dummy
     DELETE  Dummy
 
 
  • adamali's avatar
    adamali
    7 years ago

    Thanks Richie,

     

    I gave it a go and came up with the above which working perfectly fine with a simple event groovy script. I even created a The whole project which also got tons of other automated features can be downloaded from here https://earth2.digital/GenericProject-readyapi-project.xml

     

    The project has simple 2 test cases:

    1. To authenticate and get a token

    2. Execute test cases by getting all parameters from a grid (can be modified to get from excel sheet). When I release v2 I will have it reading from an excel sheet.

     

     

    Here is the Event groovy script:

     

     

    // Check if the current test step is ScenarioDataSource
    if ( testStepResult.getTestStep().getName().equals("Start")) {
    if (context.expand( '${ScenarioDataSource#method}').equals("POST")) {
    testRunner.gotoStepByName( "POSTRequest")
    } else if (context.expand( '${ScenarioDataSource#method}').equals("GET")) {
    testRunner.gotoStepByName( "GETRequest")
    } else if (context.expand( '${ScenarioDataSource#method}').equals("PUT")) {
    testRunner.gotoStepByName( "PUTRequest")
    } else if (context.expand( '${ScenarioDataSource#method}').equals("DELETE")) {
    testRunner.gotoStepByName( "DELETERequest")
    }
    } else if ( testStepResult.getTestStep().getName().equals("POSTRequest") ||
    testStepResult.getTestStep().getName().equals("GETRequest") ||
    testStepResult.getTestStep().getName().equals("PUTRequest") ||
    testStepResult.getTestStep().getName().equals("DELETERequest") ) {
    // Run the first test step in the loop
    testRunner.gotoStepByName( "ScenarioDataSourceLoop")

    //log.info ( "ScenarioDataSourceLoop" )
    }

  • adamali's avatar
    adamali
    7 years ago

    Hi richie

     

    I have published the framework with lots of details on Github. Please have a look and let me if addresses your concern about complex cases.

     

    All you need to do is to fill in the excel sheet for your test scenario.

     

    Github Repo: https://github.com/earth2digital/automated-api-ddt-framework

    LinkedIn Article: https://www.linkedin.com/pulse/how-boost-performance-your-apis-part-i-adam-ali-/

    My blog Article: https://www.earth2.digital/blog/How-to-boost-the-performance-of-your-APIs-1-adam-ali.html

     

    Cheers,

    adam

  • richie's avatar
    richie
    Community Hero
    7 years ago

    Hi,

     

    I can't help per se as i think this would require extensive groovy (some of the other people on the forum may be able to help with that) - but I'm intrigued!  I never thought about using SoapUI this way.

     

    It's a neat idea - it means the Test Analysts don't necessarily need any SoapUI experience, they can do their analysis and prep populating the spreadsheet accordingly and you only need one SoapUI Test Analyst to maintain the SoapUI projects.

     

    I apologise for the lengthy post - but I just want to highlight that this approach your thinking about seems to need a "one size fits all" approach to your test case hierarchy though and each time your test cases require a different hierarchy the event handling/groovy will require tailoring.

     

    I've been working on my current contract with SoapUI for about 18months now and I've worked on 3 different projects in that time and for each project I've had to setup my test cases differently relative to the technical and business requirements.

     

    The testing I've been doing is pretty straightforward - the projects have had the following architecture:

     

    3rd party >> ESB >> CRM

    CRM  >> ESB >> 3rd party

    3rd partyA >> ESB >> CRM >> 3rd party B >> ESB  >> CRM

     

    For these 3 projects I've had to use approx 50 different APIs and each time my test case hierarchy has had to differ to exercise those APIs to enable my testing.

     

    For example, 

     

    For the first project '3rd party >> ESB >> CRM' project I worked on - I had the following test hierarchies as follows:

     

     

    Datasource
POST Request
Prop Transfer
Properties
GET Request
Datasource Loop

Datasource
POST Request
Datasource Loop

Datasource
POST Request

Datasource
GET Request
Datasource
Property Transfer
POST Request
Delay
GET Request
Datasource Loop

POST Request
Property Transfer
Properties
Delay
GET Request

Datasource
POST Request
Groovy
Properties
Delay
GET Request

Datasource
SOAP Request
Groovy
Properties
JDBC Request

     

    That's 7 different test case hierarchies created to exercise the APIs I needed to hit to test my requirements and that was just for the 1 project alone.  I think a tester will naturally try and use the same approach each time to his test case design in soapUI because that's how you've done it before - but as I say - I've had to alter my approach depending on the API I'm testing, the purpose of the test etc., etc.

     

     

    Ok - I'll stop talking now cos I'm sure you're bored by it all - I just needed to highlight that I think your idea (which is cool) could only work if your test case hierarchies are pretty much identical.  Each time your test cases require a new test step type (or even an order change - if you currently specifiy POST, GET, PUT, DELETE - what happens if you need to change the order or have multiple instances of a single http method within the test case?) - this will require altering the events/groovy you need.  I would've thought you would need an in house SoapUI expert on hand to alter the events handler/groovy each time your test case hierarchy alters.  If you havent got the SoapUI groovy expert already to set this up I'm unsure how you're going to be able to maintain it all when you need to change the code when you're test cases alter.

     

    Just my opinion and  I could be totally wrong though!

     

    Cheers,

     

    richie

     

    • adamali's avatar
      adamali
      Occasional Contributor
      7 years ago

      Thanks Richie,

       

      I gave it a go and came up with the above which working perfectly fine with a simple event groovy script. I even created a The whole project which also got tons of other automated features can be downloaded from here https://earth2.digital/GenericProject-readyapi-project.xml

       

      The project has simple 2 test cases:

      1. To authenticate and get a token

      2. Execute test cases by getting all parameters from a grid (can be modified to get from excel sheet). When I release v2 I will have it reading from an excel sheet.

       

       

      Here is the Event groovy script:

       

       

      // Check if the current test step is ScenarioDataSource
      if ( testStepResult.getTestStep().getName().equals("Start")) {
      if (context.expand( '${ScenarioDataSource#method}').equals("POST")) {
      testRunner.gotoStepByName( "POSTRequest")
      } else if (context.expand( '${ScenarioDataSource#method}').equals("GET")) {
      testRunner.gotoStepByName( "GETRequest")
      } else if (context.expand( '${ScenarioDataSource#method}').equals("PUT")) {
      testRunner.gotoStepByName( "PUTRequest")
      } else if (context.expand( '${ScenarioDataSource#method}').equals("DELETE")) {
      testRunner.gotoStepByName( "DELETERequest")
      }
      } else if ( testStepResult.getTestStep().getName().equals("POSTRequest") ||
      testStepResult.getTestStep().getName().equals("GETRequest") ||
      testStepResult.getTestStep().getName().equals("PUTRequest") ||
      testStepResult.getTestStep().getName().equals("DELETERequest") ) {
      // Run the first test step in the loop
      testRunner.gotoStepByName( "ScenarioDataSourceLoop")

      //log.info ( "ScenarioDataSourceLoop" )
      }

      • richie's avatar
        richie
        Community Hero
        7 years ago

        Hi,

         

        Looks good!

         

        I was wondering how you are going to expand this to more complex test case scenarios?

         

        Also currently you have a single assertion checking you get a http 200 response - how are you going to handle when you need different assertions for the same http method?  I cant see how you'd do this other than create a different test case which is likely to multiply up your number of individual test cases within your test suite?

         

        Or am I misunderstanding?

         

        Cheers,

         

        richie