Forum Discussion

jkrier's avatar
jkrier
Regular Contributor
8 years ago

How to loop Test Suites or Test Cases

I have a Project that is currently built as Project Test Suite Test Case Test Steps   I have many Test Steps at the top that run queries and get log in credentials and then the remaining Test S...
  • Radford's avatar
    Radford
    8 years ago

    I think you are correct when you say "individual method Test Steps to be their own Test Case", I've always approached things assuming that a test case should be a self contained unit that can be run independently, testing one specific thing.

     

    When faced with an issue like yours, I've always created a "Common Library" Test Suite, where I create all of my reusable "common" test cases, and then in my actual test cases, use the Run TestCase Test Step to call these common test cases. These "common" test cases can return properties as required.

     

    This structure I am suggesting would look something like:

     

    Project

    |

    |--Test Suite 1

    |  |--Test Case 1

    |  |--Test Case 2

    |

    |--Test Suite 2

    |  |--Test Case 1

    |  |--Test Case 2

    |

    |--Test Suite Common Library

       |--Test Case To Do Set up Things

     

    Where all of the test cases in Test Suite 1 and 2 would call the common library test case "Test Case To Do Set up Things".

     

    Finally I alway disable my "Common Library" test suite so it is never "run" as a stand alone test suite, but you can still quite happily call its test cases.

     

    As a side note your question made me think about the TestCase options? In particular the two options:

     

    • Abort on Error
    • Fail Test Case on Error

    Have you investigated these to see if they help you?.

Radford's avatar
Radford
Super Contributor

I think you are correct when you say "individual method Test Steps to be their own Test Case", I've always approached things assuming that a test case should be a self contained unit that can be run independently, testing one specific thing.

 

When faced with an issue like yours, I've always created a "Common Library" Test Suite, where I create all of my reusable "common" test cases, and then in my actual test cases, use the Run TestCase Test Step to call these common test cases. These "common" test cases can return properties as required.

 

This structure I am suggesting would look something like:

 

Project

|

|--Test Suite 1

|  |--Test Case 1

|  |--Test Case 2

|

|--Test Suite 2

|  |--Test Case 1

|  |--Test Case 2

|

|--Test Suite Common Library

   |--Test Case To Do Set up Things

 

Where all of the test cases in Test Suite 1 and 2 would call the common library test case "Test Case To Do Set up Things".

 

Finally I alway disable my "Common Library" test suite so it is never "run" as a stand alone test suite, but you can still quite happily call its test cases.

 

As a side note your question made me think about the TestCase options? In particular the two options:

 

  • Abort on Error
  • Fail Test Case on Error

Have you investigated these to see if they help you?.

jkrier's avatar
jkrier
Regular Contributor
8 years ago

Hi Radford, thanks for replying.

 

I like your suggestion of having a common Test Suite that can be used by the Test Cases. I have been giving this some thought and I think I know how I can do this.

 

As for the Abort/Fail on error; I am not sure how I could use that since I don't want to abort the run if one step fails. The remaining tests would not be ran and so the entire "test" would be incomplete. I do have it set to "fail on error" but that is why if any iteration of the methods returns an error the entire test fails at 0% success. Unless there is some other way to use those options that I am unaware of I don't think they can help. I am always open to suggestions.

  • Radford's avatar
    Radford
    Super Contributor
    8 years ago

    When I mentioned the test case options it really was an after thought as quite a few times I've found that people don't know they are there. If you have already spotted and investigated them I don't think there is anything I can add. The main point I really wanted to get across was the use of a "Common Library" test suite.

     

    One thing I will add, is that some times I've found devloping/debugging common test cases a little bit fiddley in particular with the details returned (especially if you have common test cases asserting results). So I always paste the following code into each common test case tear down script:

     

    def logPrefix = testCase.getName() + ': '

for(result in testRunner.getResults()){
	if(result.getStatus().toString() != 'OK' ){
		def failedTestStepName = result.getTestStep().getName()
		def logPrefixStep = logPrefix.trim() + failedTestStepName + ': '
		
		log.error(logPrefixStep + 'TestStep "' + failedTestStepName + '" finished with the status ' + result.getStatus().toString())

		for(testProperty in testCase.getTestStepByName(failedTestStepName).getPropertyList()){
			if(testProperty.isReadOnly()){
				log.info(logPrefixStep + 'Output property: ' + testProperty.getName() + ' = ' + testProperty.getValue())
			}else{
				log.info(logPrefixStep + 'Input property: ' + testProperty.getName() + ' = ' + testProperty.getValue())
			}
		}
		
		for(message in result.getMessages()){
			log.error(logPrefixStep + 'Error message: ' + message) 
		}
	}
}

     

    Thus on errors the test case fails (as defined by the test case option) and all the relavent details are written to the log, in particular the input properties, allowing you then to run the common test case as a stand alone test case to investigate issues.

    • jkrier's avatar
      jkrier
      Regular Contributor
      8 years ago

      So after your initial advice this is what I ended up doing. I broke my project up so it looks like this

       

      Project

      TestSuite

      TestCase(common steps)

      queries

      scripts

      test case runner

      TestCase

      TestCase

      etc.

       

      I am using a groovy script for each test case that gathers the required inputs from the queries and runs the "Run Test Case" step with a loop for however many combinations of inputs are available for that given method (Test Case).

       

      This is an example of one of my "groovy script runners"

       

      def stepIndex = testRunner.testCase.getTestStepIndexByName( context.currentStep.label )
      def runStep = testRunner.testCase.getTestStepAt( stepIndex + 1 ).name
      log.info runStep
      def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
      def xmlResponseHolder = groovyUtils.getXmlHolder( "queryPORTAL_ORGANIZATION#ResponseAsXML" )
      def responseAsXml = context.expand( '${queryPORTAL_ORGANIZATION#ResponseAsXml#//Results[1]/ResultSet[1]}' )
      log.info "queryPORTAL_ORGANIZATIONResponseAsXml ==> " + responseAsXml
      def int rowCount = context.expand( '${queryPORTAL_ORGANIZATION#ResponseAsXml#count(//Row)}' ).toInteger()
      log.info "The Row Count Is: " + rowCount
      for ( i = 1; i > rowCount; i++ ){
       def porg = xmlResponseHolder[ "(//Results/ResultSet/Row[ " + i + " ]/PORTAL_ORGANIZATION)" ].toString()
       log.info "Portal Organization: " + i + " " + "[ " + porg + " ]"
       def ts = new java.text.SimpleDateFormat("yyyy-MM-dd-HH:mm:ss.SSS")
       def randomString = System.currentTimeMillis()
       testRunner.testCase.testSteps[ "inputValues" ].setPropertyValue( "PORTAL_ORGANIZATION", porg.toString())
       testRunner.testCase.testSteps[ "inputValues" ].setPropertyValue( "trackingkey", randomString.toString())
       testRunner.testCase.testSteps[ "inputValues" ].setPropertyValue( "timestamp", ts.toString())
       testRunner.runTestStepByName( runStep )
       
      }

      stepIndex = stepIndex + 2
      testRunner.gotoStep( stepIndex )

      Per your last response. Currently what I do for that is I have an Event (TestRunListener.afterStep) that logs the request/response and status.

       

      def testStepName = context.currentStep.label
log.info testStepName + " REQUEST " + context.currentStep.getPropertyValue("RawRequest")
log.info testStepName + " RESPONSE " + context.currentStep.getPropertyValue("RawResponse")
log.info testStepName + " TestStep has status " + "[" + testStepResult.status + "]"

      I plan on trying your script out though. Gathering the properties sounds smart to me.

       

      Thanks for your all of your help.