Forum Discussion

hindza's avatar
hindza
New Contributor
7 years ago

Help adding Assertions with Groovy Script

Hi gents, 

 

First time poster here, I am in desperate need of your help. I've tried scouring the web trying to find a simple solution to my problem, but haven't been able to find it. I am a total SoapUI noob, and what little I know has been self-taught.  

First off let me put you in perspective of why I need a quicker solution for my problems. I'll try not to provide too much details as I'm not sure what I am allowed to talk about/post.

 

I've been given a task of curating a rather large (at least in my mind) library of REST tests. I have multiple SoapUI projects which contain 10 Test Cases each (one for each user), which further contain between 50 and 300 Test Steps Each. 

 

A Project has the following structure: 

- Project Name: 

      - Test Case Alpha User: 

            - Authentication Step

            - Property Transfer Step

            - Test Step With a Name

            - Test Step With a Name

            - Test Step With a Name

            - Test Step With a Name

            - etc...

      - Test Case Beta User: 

            - Authentication Step

            - Property Transfer Step

            - Test Step With a Name

            - Test Step With a Name

            - Test Step With a Name

            - etc...

      - Test Case etc... 

 

The Project has 11 Custom Properties: 
 - DB Name property

 - User Auth. Token Alpha

 - User Auth. Token Beta

 - etc. 

 

Steps are configured like this: 

The Authentication step contains a pointer to the DB Name property and contains User Name/ID, User Password and some filter values, and returns (among other things) an Auth. Token.

The Property Transfer step takes that Auth. Token and writes it into the appropriate Custom Property (Let's say "User Auth. Token Alpha")

A Test Step contains a field that points to the DB Name property, a field that points to the User Token property and some filters. 

For Assertions, I used the "Contains" type to input an exact string I would receive as a part of the Raw Response (different for each test). 

 

Example of Raw Response: 

 

 

HTTP/1.1 200 OK
Content-Security-Policy: default-src *.environment.com 'unsafe-inline' 'unsafe-eval'
Content-Type: application/json
Date: Thu, 01 Mar 2018 01:01:01 GMT
Referrer-Policy: no-referrer-when-downgrade
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Powered-By: Servlet/2.5 JSP/2.1
X-XSS-Protection: 1; mode=block
transfer-encoding: chunked
Connection: keep-alive


{loooooong string that is actually used in the Assertion}

 

 

So, the problem is that the string can change when there is some new development on the REST API's, which invalidates all my tests which were working so far. 

 

My question is: 

Is there an easy way I can use Groovy Script to create Assertions in my Projects? 

I was thinking about something like: 

- Run All Test Cases without Assertions (which will give me the needed Responses)

- Create a Groovy Script step which will take the string from the Raw Response from each of the Test Steps and make an Assertion for each of them.

- Disable the Groovy Script step

- Test if all Test Steps Pass

- Delete the Groovy Script step

- Rinse and repeat for all Test Cases and all Projects

 

I am ashamed to admit this, but doing this manually will take me two months and it's just not feasible to do it every time there's a change in the API's. 

 

I something like this doable?  

If you need more info or details, please let me know, I will provide whatever I can just to get this sorted out. 

 

Thanks in advance to the SoapUI community!

  • hindza's avatar
    hindza
    7 years ago

    I've managed to find an answer for my problem. 

    I've also added two new assertions, one for adding Valid HTTP Status Codes and another for Invalid, as I learned this is a good practice when testing REST API's. 

    //This part adds the Invalid HTTP Status Codes in an Assertion targeting the named Test Step.
//Status codes can be added or removed as it suits your needs.
def InvalidAssertion = testRunner.testCase.testSteps["Test Step Name"].addAssertion("Invalid HTTP Status Codes");
java.lang.reflect.Field InvalidStatusAssertionCodesField = InvalidAssertion.getClass().getDeclaredField("codes");
InvalidStatusAssertionCodesField.setAccessible(true);
InvalidStatusAssertionCodesField.set(InvalidAssertion, "400,401,402,403,404,500")

//This part adds the Valid HTTP Status Codes in an Assertion targeting the named Test Step.
//Status codes can be added or removed as it suits your needs.
def ValidAssertion = testRunner.testCase.testSteps["Test Step Name"].addAssertion("Valid HTTP Status Codes");
java.lang.reflect.Field ValidStatusAssertionCodesField1 = ValidAssertion.getClass().getDeclaredField("codes");
ValidStatusAssertionCodesField.setAccessible(true);
ValidStatusAssertionCodesField.set(ValidAssertion, "200")

//This part takes the Request from the targeted Test Step and displays it onscreen as a string.
//Then it adds an Assertion of the "Contains" type and inserts the string (from the previous step) into the targeted Test Step.
def Request = testRunner.testCase.getTestStepByName('Test Step Name').testRequest.response.contentAsString
log.info(Request)
def Assertion = testRunner.testCase.testSteps["Test Step Name"].addAssertion("Contains")
Assertion.setToken(Request)

    This now constitutes a "block" of code that can be used for each of the Test Steps. 
    However, I have 1300 Test Steps and can't change every one by had. I used an excel sheet to create a list of these blocks and just copy the needed blocks into a Groovy Step. Then I run all the Test Steps, to get the Responses, then run the Groovy Step and it fills out all the Test Steps with corresponding data. The Groovy Step can be manually executed even if it's disable, which is great!

    After all of that I run the whole Test Case or Test Suite and look if everything is as expected. 

     

    If you need more details on creating a block list please ask, I won't post it now as it'll take too much time, and is not relevant to the topic.  

     

    Thanks for reading. 

     

  • hindza's avatar
    hindza
    New Contributor
    7 years ago

    I've managed to find an answer for my problem. 

    I've also added two new assertions, one for adding Valid HTTP Status Codes and another for Invalid, as I learned this is a good practice when testing REST API's. 

    //This part adds the Invalid HTTP Status Codes in an Assertion targeting the named Test Step.
//Status codes can be added or removed as it suits your needs.
def InvalidAssertion = testRunner.testCase.testSteps["Test Step Name"].addAssertion("Invalid HTTP Status Codes");
java.lang.reflect.Field InvalidStatusAssertionCodesField = InvalidAssertion.getClass().getDeclaredField("codes");
InvalidStatusAssertionCodesField.setAccessible(true);
InvalidStatusAssertionCodesField.set(InvalidAssertion, "400,401,402,403,404,500")

//This part adds the Valid HTTP Status Codes in an Assertion targeting the named Test Step.
//Status codes can be added or removed as it suits your needs.
def ValidAssertion = testRunner.testCase.testSteps["Test Step Name"].addAssertion("Valid HTTP Status Codes");
java.lang.reflect.Field ValidStatusAssertionCodesField1 = ValidAssertion.getClass().getDeclaredField("codes");
ValidStatusAssertionCodesField.setAccessible(true);
ValidStatusAssertionCodesField.set(ValidAssertion, "200")

//This part takes the Request from the targeted Test Step and displays it onscreen as a string.
//Then it adds an Assertion of the "Contains" type and inserts the string (from the previous step) into the targeted Test Step.
def Request = testRunner.testCase.getTestStepByName('Test Step Name').testRequest.response.contentAsString
log.info(Request)
def Assertion = testRunner.testCase.testSteps["Test Step Name"].addAssertion("Contains")
Assertion.setToken(Request)

    This now constitutes a "block" of code that can be used for each of the Test Steps. 
    However, I have 1300 Test Steps and can't change every one by had. I used an excel sheet to create a list of these blocks and just copy the needed blocks into a Groovy Step. Then I run all the Test Steps, to get the Responses, then run the Groovy Step and it fills out all the Test Steps with corresponding data. The Groovy Step can be manually executed even if it's disable, which is great!

    After all of that I run the whole Test Case or Test Suite and look if everything is as expected. 

     

    If you need more details on creating a block list please ask, I won't post it now as it'll take too much time, and is not relevant to the topic.  

     

    Thanks for reading. 

     

  • JHunt's avatar
    JHunt
    Community Hero
    7 years ago

    Hi, I have a few suggestions if you are interested.

     

    It seems you don't need a valid and invalid list. Surely if the code is not in the valid list, then it must be invalid.

     

    The Excel part seems unnecessary... instead you can just loop over all TestCases to add your assertions.

     

    testcases = project.testSuites.collect{suite -> suite.testCases}.flatten()
testcases.each { 
        testSteps.each {
        if (it instanceof RestRequestTestStep) {
            it.addAssertion("Valid HTTP Status Codes")
       }
        }
}

    Another idea would be to add a script assertion (using the same approach as above). The script for the added assertion might just call to a groovy script from file:

     

    evaluate (new File ('/somefolder/myAssertionScript.groovy'))

    Now you can just update the logic in your assertion script file and have that change automatically apply in any of the test steps that call that file.

     

    So what should that script do? Well it seems you want to have an 'update' mode and a 'test' mode to run your tests in. You could set the mode you want into a Workspace property, a project property, or a global soapui property. This way you just set the mode in one place and then run your tests as per that mode.

     

    To implement it, just add the logic in myAssertionScript.groovy, since it now applies on all of your RestRequestTestSteps and runs whenever you get a response on those steps:

     

    String filename = '/somefolder/' + testcase.name
if (project.getPropertyValue("Mode") == 'update') {
    new File( filename ).text = messageExchange.responseContent
} else {
    assert new File (filename).text == messageExchange.responseContent
}

    By the way, this is only intended as example of the concept -- I haven't defined some things and certainly have got some of the method names wrong.