Forum Discussion

sonya_m's avatar
sonya_m
SmartBear Alumni (Retired)
5 years ago

API Masterminds Talks - Your favorite metrics to measure effectiveness of API automation efforts?

Hi API testers!
 
There are many ways to measure API testing efficiency. For instance, you can measure: 
 
  • The number of defects 
  • How much code is covered by automation 
  • Test duration 
  • Etc. 
My question to you, API Masterminds - which metrics do you find the most useful in your API testing process? 
 
Also, if you are working from home, has anything changed in terms of metrics? We are all spending so much time on the calls with our colleagues that this might interfere with the testing processes, so maybe we should start counting this too.
APIMasterminds
workingfromhome
  • nmrao's avatar
    nmrao
    Champion Level 3
    5 years ago

    Sonya, you covered most.
    Since the question ask about API Testing, (not specific to automation)
    - map requirements vs test cases irrespective of automated tests and document the tests, alos known as Tracability Matrix
    - issue filed vs fixed rate.
    The above are being monitored by management.
    General comment, while no of defects motivates engineer, appreciate developers who does good job and surprise with no defects, may be rare.

  • HimanshuTayal's avatar
    HimanshuTayal
    Community Hero
    5 years ago

    sonya_m : There are many ways some of you have already covered but will be my added one:

     

    • First and most important to me is Defect slippage ratio : It should be minimal or low severity bugs.
    • Rework Effort Ratio : If there is any change in current functionality then how much time it is taking to update the existing Test Automation Project.
    • Test Case Execution Percentage - Passed and Failed Test Case Percentage per execution.,  and many more.

    And yes as you have mentioned, metrics is same but i am also spending lot of time in attending calls because in office if any of the colleague faces any issue they just directly call to me and the problem got resolved and vice versa,

    but now they used to call me and we connect over skype then screenshare and if need of anyone else is required then we call that person and connect him/her to call......lot of time is now getting wasted and hence working hours now increasing because of all these issues.