Test Case Statistics
I have developed a few test cases that use a datasource for test case ID and test case details i.e. Excel file with test case ID, test case description, input parms, expected values. For example
testcase_id testcase_desc ws-userID ws-Password o-faultcode
AUTH1 Bad security Credentials mybad mybad _ns0:FailedAuthentication
AUTH2 Missing security user ID mygood _ns0:InvalidSecurity
AUTH3 Missing security Password mygood _ns0:InvalidSecurity
However when test suite is run, the reports just say test case PASS or FAIL summary and some details when an assertion fails.
Test started at 2011-03-15 18:27:45.812
Step 1 [DataSource] OK: took 14 ms
Step 2 [GetAssets] OK: took 120 ms
Step 3 [DataSource Loop] OK: took 0 ms
-> Current row = 1
Step 4 [GetAssets] OK: took 49 ms
Step 5 [DataSource Loop] OK: took 0 ms
-> Current row = 2
Step 6 [GetAssets] OK: took 58 ms
Step 7 [DataSource Loop] OK: took 0 ms
-> Current row = 3
Step 8 [GetAssets] OK: took 24 ms
Step 9 [DataSource Loop] OK: took 0 ms
-> Current row = 4
TestCase finished with status [FINISHED], time taken = 265
Was looking for way to get information for each Loop which will be for different test cases
test case ID, test case description, status
AUTH1, Bad security Credentials, pass
AUTH2, Missing security user ID, pass
AUTH3, Missing security Password, pass
On a related note, can similar data be obtained at the assertion level? When a test case fail we see the assertion message, but also looking for both PASS and FAIL statistics. Since many assertions are groovy script based, perhaps there is a way to set something.
There were some very old posts on the topic....but looking for guidence on how to report/statistics on to best do this now.
testcase_id testcase_desc ws-userID ws-Password o-faultcode
AUTH1 Bad security Credentials mybad mybad _ns0:FailedAuthentication
AUTH2 Missing security user ID mygood _ns0:InvalidSecurity
AUTH3 Missing security Password mygood _ns0:InvalidSecurity
However when test suite is run, the reports just say test case PASS or FAIL summary and some details when an assertion fails.
Test started at 2011-03-15 18:27:45.812
Step 1 [DataSource] OK: took 14 ms
Step 2 [GetAssets] OK: took 120 ms
Step 3 [DataSource Loop] OK: took 0 ms
-> Current row = 1
Step 4 [GetAssets] OK: took 49 ms
Step 5 [DataSource Loop] OK: took 0 ms
-> Current row = 2
Step 6 [GetAssets] OK: took 58 ms
Step 7 [DataSource Loop] OK: took 0 ms
-> Current row = 3
Step 8 [GetAssets] OK: took 24 ms
Step 9 [DataSource Loop] OK: took 0 ms
-> Current row = 4
TestCase finished with status [FINISHED], time taken = 265
Was looking for way to get information for each Loop which will be for different test cases
test case ID, test case description, status
AUTH1, Bad security Credentials, pass
AUTH2, Missing security user ID, pass
AUTH3, Missing security Password, pass
On a related note, can similar data be obtained at the assertion level? When a test case fail we see the assertion message, but also looking for both PASS and FAIL statistics. Since many assertions are groovy script based, perhaps there is a way to set something.
There were some very old posts on the topic....but looking for guidence on how to report/statistics on to best do this now.