Forum Discussion

JoostDG's avatar
JoostDG
Frequent Contributor
6 years ago

SubmitListener.afterSubmit - gather all codes from http400 responses

Hi community.

 

I have a nice little script that needs some tweaking to get it working properly and would really appreciate your help.

 

Use Case: I want to verify which http400 Bad Request error codes my testcases cover. From developers I have a list of implemented codes, so it would be nice to see that my test project covers all of these. If not, then I know I miss some scenario's. An example of a bad request results typiacally in this response with one code and one or more errors.code values:

 

HTTP/1.1 400 Bad Request

{
"code": 1,
"message": "Technical failure.",
"errors": [
{
"code": 51,
"message": "This field is mandatory.",
"description": "Additional description.",
"field": "target[0].type"
},
{
"code": 51,
"message": "This field is mandatory.",
"description": "Additional description",
"field": "target[1].type"
},
{
"code": 83,
"message": "Other message.",
"description": "",
"field": ""
}
]
}

 

So I am interested in all above code values.

 

My piece of groovy script for SubmitListener.afterSubmit at event level works fine when I do a run at test step level, but when I run from test case level (or test suite or project) it goes wrong. The context.expand below does no longer returns the expected value 1, but "" skipping the step that pushes this value to the listArray. Same for those at errors.code level. So it seems that depending on the context where the test is run the context.expand is not the same.

The values for my status (responseheader) and for my stepName (submit.request.getName()) does always provide met the correct test tesult I can see in my logs, irregardles of run context. So I assume I have to find a way to replace the context.expand with a submit.getResponse() somehow? But how do I get my code and errors.code values?

 

Mighty thanks in advance for anyone who might guide me to help this issue!!!

 

import com.eviware.soapui.support.JsonUtil
import groovy.json.JsonSlurper

String endpoint = submit.request.getEndpoint()
if (endpoint.contains("myEndPointUnderTest.be")) {
        // errorCodeList is defined as empty array [] in my project setup script
        def listProperty = context.expand('${#Project#errorCodeList}')
        def listArray = new JsonSlurper().parseText(listProperty)
        String status = submit.getResponse().getResponseHeaders() "#status#"
        //log.info "response for endpoint "+ endpoint + " from submitListener event handler  " +status
        if (status.contains("400")) {
            def stepName = submit.request.getName()
    		  log.info ("stepName: " + stepName)
		  def code = context.expand('${'+stepName+'#Response#$[\'code\']}')
		  if (code != ""){
		  log.info ("Code to add :" + code)
            listArray.push(code)}
            def errors = context.expand('${'+stepName+'#Response#$[\'errors\'][*]}')
            if (errors != "") {
                def numberOfErrorCodeArray = JsonUtil.parseTrimmedText(errors)
                int numberOfErrorCodes = numberOfErrorCodeArray .size()
                for (i = 0; i < numberOfErrorCodes ; i++) {
                    def errorCode = context.expand('${'+stepName+'#Response#$[\'errors\'][' + i + '][\'code\']}')
                    listArray.push(errorCode)
                    log.info "and for this step : " + stepNaam + " we add following error codes " + errorCode +" to the listArray"
                    log.info "total= " + listArray
                }
            }
        }
        def step = submit.getRequest()
        log.info("final listArray = " + listArray)
        def testcase = step.getTestCase()
        def testsuite = testcase.getTestSuite()
        def project = testsuite.getProject()
        project.setPropertyValue("errorCodeList", listArray.toString())

FYI: In my project teardown I gather all these values, sort and get the unique values:

 

log.info "------ HTTP400 ERROR CODES covered ------"
def listProperty = context.expand('${#Project#errorCodeList}')
def listArray = new JsonSlurper().parseText(listProperty)
// sort Array
listArray.sort()
int n =  listArray.size()
int res = 0
def uniqueList = []
for (i=0;i<n;i++){
while (i< n-1 && listArray[i] == listArray[i+1]){
	i++
	}
	res ++
	uniqueList.push(listArray[i])
}
log.info (res+ " unique (error)codes checked in my project run. This is the full list covered: "+ uniqueList.toString())

 

 

 

REST
Scripting
submitListener

2 Replies

Sort By
  • JoostDG's avatar
    JoostDG
    Frequent Contributor
    6 years ago

    Hmmm... my reply got tagged as spam and deleted for some reason...

     

    Anyway, here is the summary again: 

     

    Found the answer after good night of sleep...indeed I had to replace the context.expand to make it work at al levels.

     

     

    //event makes list of test coverage on all http 400 (error)codes

import com.eviware.soapui.support.JsonUtil
import groovy.json.JsonSlurper

String endpoint = submit.request.getEndpoint()
if (endpoint.contains("mydesired.endpointUnderTest.com")) {
    try {
        def listProperty = context.expand('${#Project#errorCodeList}')
        def listArray = new JsonSlurper().parseText(listProperty)
        String status = submit.getResponse().getResponseHeaders() "#status#"
        if (status.contains("400")) {
            def stepName = submit.request.getName()
            //def code = context.expand('${'+stepName+'#Response#$[\'code\']}') // context.expand does not work for SubmitListener.afterSubmit when run context differs from test step/request level (e.g. test case, project,...). Code value = "" in those cases
            def json = new JsonSlurper().parseText(submit.getResponse().contentAsString)
            def code = json.code
            if (code != "") {
                //log.info ("Code to add :" + code)
                listArray.push(code)
            }
            def errors = (json.errors)
            if (errors != "") {
                int numberOfErrorCodes = errors.size()
                for (i = 0; i < numberOfErrorCodes ; i++) {
                    def errorCode = json.errors[i].code
                    listArray.push(errorCode)
                    // log.info "For this step : " + stepName + " we have following error codes " + errorCode +" added to the list. Total now is = " + listArray
                }
            }
        }

// keep unique values (instead of doing this in the project teardown)
// first sort Array
        listArray.sort()
        int n = listArray.size()
        int res = 0
        def uniqueList = []
        for (i = 0; i < n; i++) {
            while (i < n - 1 && listArray[i] == listArray[i + 1]) {
                i++
            }
            res++
            uniqueList.push(listArray[i])
        }

        def step = submit.getRequest()
        // log.info("Final listArray = " + uniqueList)
        def testcase = step.getTestCase()
        def testsuite = testcase.getTestSuite()
        def project = testsuite.getProject()
        project.setPropertyValue("errorCodeList", uniqueList.toString())
    }
    catch (Exception ex) {
        log.warn "SubmitListener.afterSubmit - getAllHttp400CodeAndErrorCodes goes in exception and will do nothing..."
    }
}
    • JoostDG's avatar
      JoostDG
      Frequent Contributor
      6 years ago

      At the risk of (really) spamming my own question:

      A new and improved script with some backups and some new functionality: Cover now ALL http error codes from range 401 to 404. Split up list between code and error.code. Log warnings in case we do not provide an error paylod message in the response. 

       

       

      //event makes list of test coverage on all http (error)codes

import com.eviware.soapui.support.JsonUtil
import groovy.json.JsonSlurper

def step = submit.getRequest()
def testcase = step.getTestCase()
def testsuite = testcase.getTestSuite()
def project = testsuite.getProject()
def tsname = testsuite.getName()

String endpoint = submit.request.getEndpoint()
if (endpoint.contains("mydesired.endpoint.com") && tsname != "_Authentication" && tsname != "Info") {
    try {
        //def listCodeProperty = context.expand('${#Project#codeList}')
        def listCodeProperty = project.getPropertyValue("codeList")
        def listArrayCode = new JsonSlurper().parseText(listCodeProperty)
        //def listErrorCodeProperty = context.expand('${#Project#codeErrorList}')
        def listErrorCodeProperty = project.getPropertyValue("errorCodeList")
        def listArrayErrorCode = new JsonSlurper().parseText(listErrorCodeProperty)
        String status = submit.getResponse().getResponseHeaders() "#status#"
        if (status.contains("400") || status.contains("401") || status.contains("403") || status.contains("404")) {
            def stepName = submit.request.getName()
           
            def response = submit.response.getContentAsString()
            if (response != null) {
                def json = new JsonSlurper().parseText(response)
                def code = json.code
                if (code != "" && code != null) {
                    //log.info ("Code to add :" + code)
                    listArrayCode.push(code)
                }
                def errors = (json.errors)
                if (errors != "" && errors != null) {
                    int numberOfErrorCodes = errors.size()
                    for (i = 0; i < numberOfErrorCodes ; i++) {
                        def errorCode = json.errors[i].code
                        listArrayErrorCode.push(errorCode)
                        // log.info "and for this step : " + stepName + " there are following error codes " + errorCode +" added. Total is now = " + listArrayErrorCode
                    }
                }
            } else {
                log.warn("For teststep " + stepName + " -  testcase  " + testcase.getName() + " - testsuite: " + testsuite.getName() + " is a " + status + " WITHOUT error payload! Detected by SubmitListener.afterSubmit event getAlleHttpCodeandErrorCodes. ")
            }
        }

// unique codes
// sort ArrayCode
        listArrayCode.sort()
        int n = listArrayCode.size()
        int numberOfUniqueCode = 0
        def uniqueListCode = []
        for (i = 0; i < n; i++) {
            while (i < n - 1 && listArrayCode[i] == listArrayCode[i + 1]) {
                i++
            }
            numberOfUniqueCode++
            uniqueListCode.push(listArrayCode[i])
        }

// unique ErrorCodes
// sort ArrayErrorCode
        int m = listArrayErrorCode.size()
        listArrayErrorCode.sort()
        int numberOfUniqueErrorCode = 0
        def uniqueListErrorCode = []
        for (i = 0; i < m; i++) {
            while (i < m - 1 && listArrayErrorCode[i] == listArrayErrorCode[i + 1]) {
                i++
            }
            numberOfUniqueErrorCode++
            uniqueListErrorCode.push(listArrayErrorCode[i])
        }

// save to project properties (if values are offered)
        if (uniqueListCode.toString() != "[null]") {
            project.setPropertyValue("codeList", uniqueListCode.toString())
            project.setPropertyValue("numberOfUniqueCodes", numberOfUniqueCode.toString())
        }
        if (uniqueListErrorCode.toString() != "[null]") {
            project.setPropertyValue("numberOfUniqueErrorCodes", numberOfUniqueErrorCode.toString())
            project.setPropertyValue("errorCodeList", uniqueListErrorCode.toString())
        }
    }
    catch (Exception ex) {
        log.warn "SubmitListener.afterSubmit - getAlleHttpCodeAndErrorCodes goes in exception en thus does nothing..."
    }
}