cancel
Showing results for 
Search instead for 
Did you mean: 

TestExecute fails every region checkpoint on VM

Highlighted
Occasional Contributor

TestExecute fails every region checkpoint on VM

Hi,

 

I am running tests on a VM using TestExecute. I am running it at the same resolution as the main test machine I use to create them. And all browser settings are the same. The comparisons also look the same.

I am not sure what else to check.

6 REPLIES 6
Highlighted
Community Hero

Re: TestExecute fails every region checkpoint on VM

Different OS?  Different color settings?

Image comparison checkpoints are NOTORIOUS for being imprecise.  When doing the checkpoint, the log should show a "mask" showing, in red, the different pixels.  Anything stand out?  Is there a "date" or "time" on screen that is different each time?  Lots of things could cause a difference... even if everything is exact, there's always a little bit that's off...


Robert Martin
[Community Expert Group]
Please consider giving a Kudo if I write good stuff
----

Why automate?  I do automated testing because there's only so much a human being can do and remain healthy.  Sleep is a requirement.  So, while people sleep, automation that I create does what I've described above in order to make sure that nothing gets past the final defense of the testing group.
I love good food, good books, good friends, and good fun.

Mysterious Gremlin Master
Vegas Thrill Rider
Extensions available
Highlighted
Community Leader

Re: TestExecute fails every region checkpoint on VM

Yes, its hard to make image comparison robust and reliable. But still achievable.

 

Some hints:

- create region reference image on the same computer the test run

- use mask

- use pixel and color tolerance

Un sourire et ça repart

Highlighted
Contributor

Re: TestExecute fails every region checkpoint on VM

I guess you have a different color depth or windows themes.

So, the region checkpoint doesn't lie. Pictures are really different.

Highlighted
Community Manager

Re: TestExecute fails every region checkpoint on VM

Thank you Community for great suggestions here.

 

@kylegogtp were you able to solve this using the suggestions? Please share.


Sonya Mihaljova
Community and Education Specialist

 Join the virtual SmartBear Connect user conference on April 27-28! Get your free virtual ticket today!
Occasional Contributor

Re: TestExecute fails every region checkpoint on VM

I haven't been able to try yet. I started moving tests to TestExecute because of my other issue I posted about chromium edge. The VM still had the "legacy" edge. I reinstalled windows on my test machine and have been getting the tests to work on the original machine again.

Highlighted
Community Manager

Re: TestExecute fails every region checkpoint on VM

Thank you for the update @kylegogtp , keep us posted and the Community will see how esle we can help.


Sonya Mihaljova
Community and Education Specialist

 Join the virtual SmartBear Connect user conference on April 27-28! Get your free virtual ticket today!