Forum Discussion

nifengfei's avatar
nifengfei
Occasional Contributor
14 years ago

Code coverage seems not correct

hi experts,



I use AQtime to do the code coverage for my applications using my tests, I do not that trust about the result. then i write a simple code to test it. It seems that the code coverage profile is wrong whatever by line or by routine level.



the following is my steps:



1. using vs2008 to create a Win32 Console Application using c++;

2. put the following codes:

***codes start***


#include "stdafx.h"





class Hello

{

private:

int x;

int y;

public:

Hello()

{

x=10;

y=10;

}

int Add()

{

return x+y;

}

int Sub()

{

return x-y;

}

~Hello()

{

}

};





int _tmain(int argc, _TCHAR* argv[])

{

Hello *aHello= new Hello();

return aHello->Sub();

}
***codes end***

3. build the above project in debug mode

4. create an AQtime project, select coverage profile, add the above exe as the module to be profiled, check "Full Check by Lines" & "by Lines", and click run.

5. after finishing the profile, result is: by line 90.91%, by routine 100%!!! 



obviously, in my code, i didnot call the function Add, but why the result is 100%.



further, i add another constructor for this class as following:

Hello(int x, int y)

{

    this->x=x;

    this->y=y;

}



build->using AQtime to profile, the result is the same as the previous. by line is 90.91%!!! why I added another meaningful codes, but the same calls, the by line result doesnot change?



so my question is:

1. Is the result generated by AQtime correct?

2. If it is, how does AQtime to count the base line number, base routine number and the hit lines, routines?



Looking forward to your reply. thanks.

5 Replies

  • AlexKaras's avatar
    AlexKaras
    Champion Level 3
    Hi,



    I did not try your code yet, but a quick idea: It is possible that smart optimizing compiler removed unused routines from the resulting exe. What if you try to generate map file or something like it and look whether the unused functions are in the exe?
  • AlexKaras's avatar
    AlexKaras
    Champion Level 3
    Hi again,



    Been curious, I compiled your code (in VS2008) and looked what was produced in AQtime's disassembler... It looks like that the final binary was seriously optimized :) (see attached screenshot), so results you've got look like correct one...
  • nifengfei's avatar
    nifengfei
    Occasional Contributor
    Hi Alex,



    as you said, there will be some optimization in compiling codes, so we can not calculate the base code lines & routines via counting the code lines & routines in source files like .cpp?



    but i want to know if there is any rules or methods that we can calculate the base code lines & routines through the source code files? that's, how does AQtime calculate the profile result? how does AQtime decide the base code lines & routines of compiled modules? thanks.
  • AlexKaras's avatar
    AlexKaras
    Champion Level 3
    Hi Feng,



    If I got your question right...

    It is my understanding that AQtime uses debug information to match source code lines to binary code and all calculations are based on the debug information.

    There should be tools that collect statistics on the source code (excluding empty lines and comments, etc.) but this is not what is provided by AQtime.
  • aqAnt's avatar
    aqAnt
    SmartBear Alumni (Retired)

    Hi Feng,





    In addition to Alex's answer, please note that AQtime calculates line coverage by counting coverage of binary code blocks. You can see the "Hit Block Count" and "Block Count" values displayed for each line in the Details panel. Please refer to the Coverage Profiler - Overview article for details.