How to capture Test PASS or FAIL test count using a property

Oh, actually, I forgot I think there is also a project level runner and teardown script. I'll try to adapt the script to work there, maybe tomorrow. That should hopefully be a lot cleaner.

Awesome, that did worked.

 I thought that the variable was at project level, so it will gather all the stats for the project, which has multiple testsuites.

 I wanted to get entire count of the tests for the Project.

 So I was trying to use that as TearDown script at Project level and hence was getting error.

 Is there a magic way to get count {sum} for all the tests from all the suites within a project?

Again thank you for your help

That worked perfectly as you said.

 Some of my Projects have 50-100 suites...

 so I guess, I need to add the teardown to all the suites..

As time permits I can do that if its just one time thing...

I wish there is just a better way via some soapUI property to grab the numbers from Project transaction Log itslef.

I see that transaction log keeps count of "steps pass" "steps fail". If that can be extracted, that would do the job in some cases.

Thank you for help again..

Hi,

ok, glad it worked ok.

Yes, I felt when I suggested this solution that its not very direct or scalable. I haven't had time to check, but like you said, I would imagine that the statistics can be grabbed from somewhere else or in a different way in one go, will have a check when I get a chance... As a more hacky alternative, since the tear down script is the same for each TestSuite, we could probably write a quick script to automatically copy it into every TestSuite's TearDown script in the whole project - would this be helpful to you?

Regards,

Rup 

Sure, it will be handy to have such a script as well.

However no rush, if time permits and its something easier for you to do then only go for it.

Otherwise no rush..

Thank you

Hi,

Ok, no problem - maybe have a try of this project level TearDown script:

def passed=0
def failed=0

for (testSuiteRunner in runner.results ) 
{
	for ( testCaseRunner in testSuiteRunner.results )
	{
    		for ( testStepResult in testCaseRunner.results )
    		{
    	 		if (testStepResult.status.toString()=="OK") passed++
    	 			else failed++
		}	
	}
}
log.info "passed="+passed+" failed="+failed

runner.project.setPropertyValue("passed", "$passed" ) 
runner.project.setPropertyValue("failed", "$failed" ) 

It's much like the other one, except it uses the project's runner object to get access to the results from all TestSuites contained in the project. Apart from only needing one script in one place, it can also reset the properties to zero on each project level run. Note - remember to remove any other TearDown scripts that we have been trying!

Just incase you haven't see the location of the project level TearDown script (as I had missed it), here's a screen shot: 

Regards,

Rup

Thanks Rup,

 it worked very well, however it is initializing the failed count to '1'

 So even if the suite has 0 failures, its reporting 1 all the time

 wonder why is that

Thanks!

Ok, cool.

Hmm.. I can only imagine one of the TestSteps somewhere is returning a status other than OK, then the if statement:

if (testStepResult.status.toString()=="OK") passed++
    	 			else failed++

Would increment failed.

Looking at the possibilities https://www.soapui.org/apidocs/com/eviware/soapui/model/testsuite/TestStepResult.TestStepStatus.html

We have:

So perhaps we should change the if statement to something like:

if (testStepResult.status.toString()=="OK") passed++
    	 			else if (testStepResult.status.toString()=="FAILED") failed++
    	 				else log.info "test step: "+testStepResult.testStep.name+" returned status: "+testStepResult.status.toString()

I haven't tested this, but if should only increment failed when that status is returned, and in the unexpected case, should tell us which TestStep returned the unexpected status 

Maybe give this a try?

Thanks,

Rup

Ok, let me try

 and we sure it does not count the disabled suites as failure?