Forum Discussion

ChiefBrian's avatar
ChiefBrian
Contributor
9 years ago

Getting pixel value from Picture object is incredibly slow

I am trying to create my own custom image comparison algorithm. The first, what I thought, simple step would be to iterate over the each pixel in each image and compare them.  This however is so slow that I have yet to wait long enough for it to finish.    Is this a known issue?  Any alternative ideas instead of using the builtin  TC picture object to get the pixel data?

 

My images are fairly large, 2000 x 2000 but this still seems way slower than it should be.

 

function test()
{
var pic1 = Utils["Picture"]
pic1["LoadFromFile"]("C:\\Users\\testaccount\\Documents\\Test\\Static.png")
var pic2 = Utils["Picture"]
pic2["LoadFromFile"]("C:\\Users\\testaccount\\Documents\\Test\\newTest.png")

CompareImage(pic1,pic2)
}


function CompareImage(pic1,pic2)
{
var totalErrors = 0;
var width = pic1["Size"]["Width"]
var height = pic1["Size"]["Height"]

for(var i=0; i<width;i++){
  for(var j=0; j<height; j++){
     if(pic1.Pixels(i,j) != pic2.Pixels(i,j))
       totalErrors++;  
   }
}

 

}

  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor
    9 years ago
    Something else you could do is just do a quick binary file comparison.... if the files match, then you don't need to do any more processing.

    Then, perhaps, take them in 100 x 100 chunks. If a chunk shows a mismatch, then report the problem... perhaps then do the pixel by pixel compare but really, that seems to be on the lines of building a nuclear powered can opener when a simple hand crank one will do
  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor
    9 years ago
    I guess I'm not understanding how this pixel by pixel comparison or otherwise alternative method is preferable over the existing Region.Compare method. 2000 x 2000 is literally going through 2 million pixels... twice (once for each picture) and the color data for each.... that's 4 million pixels being checked by pairs...

    There are probably other utils available out there if the existing compare functions in TC don't suffice, but certainly I wouldn't do anything like this nested for loop deal...
  • ChiefBrian's avatar
    ChiefBrian
    Contributor
    9 years ago

    The current compare method doesn't give me enough flexibility other than just ignoring a specific number pixel differences. This method results in many false positives or if  tolerance is set it too high it ends up missing actual problems.  The compare method is much much faster and must be doing something similar to what I am trying to do so it seems this should be possible.

     

    Do you have any suggestions on alternative tools or other methods that are not "this nested loop deal"?

     

    • Colin_McCrae's avatar
      Colin_McCrae
      Community Hero
      9 years ago

      "The current compare method doesn't give me enough flexibility other than just ignoring a specific number pixel differences. This method results in many false positives or if  tolerance is set it too high it ends up missing actual problems."

       

      So I'm guessing from this, you are having to tweak the tolerances to get it to match?

       

      I'm not sure that you're going to get it any better writing one yourself? What are you hoping to put in there that will make a custom version better?

       

      As tristaanogre says, it's going to be slow. The "Compare" method will be faster, I would imagine, primarily due to being written in a much lower level language than a scripted (interpreted) language in the TC IDE. I can't imagine anything scripted will ever match it for speed, no matter how well you write it.

       

      As mentioned above, you could try using something external. Or I had a quick look to see if there were any online API's that do this. And there are. But I didn't find any free ones ....

      • tristaanogre's avatar
        tristaanogre
        Esteemed Contributor
        9 years ago

        Just as an additional note... I find image comparisons to be generally a bad testing methodology for most situations anyways.  There are times when it is necessary, especially if you are doing ACTUAL image processing and need to make sure that an image after being processed is correct (like for medical imaging, etc).  But doing region or image comparisons for most desktop or web based applications when an object comparison or object property comparison will do... you've already noted the problems of tweaking tolerance and the pitfalls there... as well as it generally being imprecise all around.  You are taking a picture and comparing it to another picture and using that to determine whether or not intricate code is being executed properly.

        So, while your need as stated is that you want to compare images, I wonder if there is a better way of conducting the test that avoids image comparison all together that would achieve more accurate results and a better feedback as to the state of the AUT.

    • tristaanogre's avatar
      tristaanogre
      Esteemed Contributor
      9 years ago
      ChiefBrian wrote:

      The current compare method doesn't give me enough flexibility other than just ignoring a specific number pixel differences. This method results in many false positives or if  tolerance is set it too high it ends up missing actual problems.  The compare method is much much faster and must be doing something similar to what I am trying to do so it seems this should be possible.

       

      Do you have any suggestions on alternative tools or other methods that are not "this nested loop deal"?

       

      Quick google search found an image diff tool by Perforce, there's a commandline utility by Image Magick, guiffy software has an Image Diff Tool... so, yeah, bunch of them out there.

      However, what you'll find with all of these is that every one of them has similar stuff about setting image threshholds, tolerance, color depth, etc.... so, much like Region.Compare, you'll still need to "tweak" the settings in order to find that sweet spot.  So, no matter what you do, either with something you write custom, or if you use some other available utility, you'll run into the same pitfalls you mentioned of false postives, false negatives, etc., due to those VERY subtle differences in bitmap renderings that are always the case with any digital image.  Even if you take two screenshots of the EXACT same component on the EXACT same screen in the EXACT same version of the software, there will STILL be slight differences in pixels due to just the way things are rendered via the graphics processor of a machine... you can't get away from it, unfortunately.

       

      As Colin_McCrae said, you're attempting to do something in an interpreted language which is executed, not as a compiled executable, but via an engine that is literally reading the script line by line... it will ALWAYS be slower than any compiled tool... it's best to either use TestComplete's built in routine or one of these other utilities if speed is your concern... but regarding your desired accuracy... not sure you'll find anything.... best of luck!

      • Colin_McCrae's avatar
        Colin_McCrae
        Community Hero
        9 years ago

        If you want some other samples, in a veritable multitude of different languages, there are a pile of them here:

         

        https://rosettacode.org/wiki/Percentage_difference_between_images

         

        There are a few languages TestComplete supports in there. But, as already mentioned, they'd still be interpreted, and thus slow.

         

        Of the non-supported stuff, there are probably some in there you could tweak, compile, and link to TestComplete.

         

        But, for all the reasons already mentioned, you're not likely to gain much over the built in methods.

  • ChiefBrian's avatar
    ChiefBrian
    Contributor
    9 years ago

    Thanks for all of the feedback. I am going to look into some of the other tools that were mentioned here.

     

    Trust me I wish we didn't have to try and use image comparison as well. We have avoided it like the plague for sometime now but there are some cases with the app we are testing that give us no other real option for testing.  The app generates 3D data on screen with shadows...etc and there is inherently some slight randomness to this.  A pixel can be moved by just a single pixel within the image. This won't be noticed by the user and is something we don't want to flag as error in our test.

     

    I am not sure if my original idea would will or not it was just something I was toying with to try and get decrease the amount of false positives. What I was going to try and do was add an additional tolerance parmenter that if an error occurred would search 1 pixel in every direction of the error pixel to see if it found the correct colored pixel.  I will have to spend some time to see if some of these other tools will work to do that at a reasonable speed.

     

    Smartbear may want to add a note to this section of the help file about speed, since this where I started.

    https://support.smartbear.com/testcomplete/docs/testing-with/checkpoints/regions/comparing-images-in-scripts.html#Custom

     

    Thanks again for the help.

    Brian

     

    • tristaanogre's avatar
      tristaanogre
      Esteemed Contributor
      9 years ago

      Yeah, sometimes it's unavoidable... but it is a pain when you do need to use it.

      Question: do you need to compare the WHOLE image or could you grab smaller chunks for verification that might exclude those "blurry" areas? That might be another solution, to limit the size and selection of the image being compared so that your tolerence factors are more predictable.