Forum Discussion

mtu's avatar
mtu
New Contributor
11 years ago

Region checkpoint fails when run on different computer

There is a problem I run into when using TestComplete concerning region checkpoints. Our company develops medical software for analyzing X-ray and MR images. We need to do visual checks on these images to make sure the analyses are correct.

This seems to be perfect for the region checkpoints. However, these tests are not developed by just one tester, but by several, which means different software and hardware configurations. The result is that region checkpoints fail (I think as a result of different video cards). When comparing the images, the comparison shows that (to me) random pixels are different all over the image. To the eye, there is no visible difference.

Does anyone else have this problem? How can I prevent the tests from failing due to this issue?
  • TanyaYatskovska's avatar
    TanyaYatskovska
    11 years ago

    Hi Marijn,


     


    As the comparison fails in random places, it's very difficult to tell TestComplete how to compare the pictures properly. You can update all checkpoints when executing the test on a computer at the first time. After that, the image comparison should pass. Also, I suggest that you pay attention to the tips described in the "Factors Affecting Image Comparison" article.


     

  • TanyaYatskovska's avatar
    TanyaYatskovska
    SmartBear Alumni (Retired)
    11 years ago

    Hi Marijn,


     


    As the comparison fails in random places, it's very difficult to tell TestComplete how to compare the pictures properly. You can update all checkpoints when executing the test on a computer at the first time. After that, the image comparison should pass. Also, I suggest that you pay attention to the tips described in the "Factors Affecting Image Comparison" article.


     

  • mtu's avatar
    mtu
    New Contributor
    11 years ago
    The color tolerance seems to help, but doesn't that increase the chance of the test passing when it is actually failed?
  • gabriel_henriqu's avatar
    gabriel_henriqu
    Occasional Contributor
    11 years ago
    Hi Marijn,



    we have some regions checkpoints in our automations. We defined the requirements to run the automations like resolutions and Windows theme, but when changing the OS (from Windows 7 to Windows XP) we are facing the same problem. I think this is a expected drawback to this test type.



    I our case, we can't use the color tolerance because the result is unpredictable.
  • mtu's avatar
    mtu
    New Contributor
    11 years ago
    Hi Gabriel,

    What is your solution? Do you use the region checkpoints and check the failed ones manually each time you execute your tests, or do you not use region checkpoints at all?

    Even though I think test results should be reproducible on any system (which they are currently not in case of region checkpoints), I am currently trying to use a shared virtual machine, such that the result is the same for anyone who executes the tests.

    I'll keep you posted on the results.
  • gabriel_henriqu's avatar
    gabriel_henriqu
    Occasional Contributor
    11 years ago
    Hi Marijn,



    I use regions checkpoints, but when running a automation, some preconditions has to be attended (like resolution config, theme, etc).



    A shared virtual machine is a good idea, but you still need establish some preconditions, like resolution (Remote Desktop has a connection configuration that enables the user select a resolution on connect). We use virtual machines (they aren't shared) to run these automations, and the tester can "develop" the automation on your own desktop (meeting the preconditions).