extend chrome test to cross-browser test: screenshots are the problem
I got a really good and stable running test (keyword tests), that checks websites in Chrome.
As our software supports also supports IE11 and lateron Edge, it is always on my mind to run the test with other browsers (IE11 or Edge) as well.
So I am thinking how I can improve my test to run with other browsers as well. I also read http://support.smartbear.com/viewarticle/70762/ some time ago.
As far as I see my test can do most stuff needed for such cross browser testing
- starting the test with different browsers > this can easily be implemented
- as our namemapping is really clean and stable the test will run stable (already checked), only some minor issues will have to be fixed
The only problem is, that we heavily rely on screenshots (as we really want to check the stuff the user is faced with).
These screenshots do look differently in different browsers. The data shown is correct, but the difference is too big to be handled with masks.
So how can I handle this?
The complete teststructure (keyword tests) can stay the same, only there have to be different baseline pictures for each browser.
1) make a copy of all the tests, one for chrome, one for ie11 >> no good solution, twice the maintenance effort
2) in front of each checkpoint checking if current browser is chrome or IE >> no good solution inserting > 1000 if-else-statements in keyword tests
Any help appreciated.
As a quick idea:
-- Put screenshots into the subfolders below the parent root (e.g.: Images\Chrome\, Images\IE11\, Images\Edge\);
-- At the beginning of the test, identify the browser and compose the correct path to screenshots (store the path in some (temporary project) variable);
-- Use this variable to reference the screenshot.
Does this help?
/Alex [Community Hero]
[Community Heroes] are not employed by SmartBear Software but
are just volunteers who have some experience with the tools by SmartBear Software
and a desire to help others. Posts made by [Community Heroes]
may differ from the official policies of SmartBear Software and should be treated
as the own private opinion of their authors and under no circumstances as an
official answer from SmartBear Software.
The [Community Hero] signature is used with permission by SmartBear Software.
I think I did not get how I should do this.
We are using keyword tests and, as far as I see, I have a problem that I cannot compose the path or name to a picture through a variable, I just can select an image from the store.
Or am I missing something?
I had some other ideas how to make this work. But all of them break at least at a workflow to update pictures when I have to because of changes to the layout.
no other idea regarding the screenshot-problem?
Is the some event fired at the start of a checkpoint? (not after passing it)
any suggestions or ideas welcome. I am open to all options as long as they work properly.
I just tested and our test really runs in all browsers, I just have to set the browser-parameter.
Somehow solving the screenshot-problem would be so huge!
As for now the best idea, that I will at least try is to manipulate the *.mds file of each project.
<Node name="item data">
<Prp name="relpath" type="S" value="Stores\Stores.tcStores"/>
I will copy the complete \Stores-folder to
and before the start of a test I will change the mds-files to
- <Prp name="relpath" type="S" value="StoresIE\Stores.tcStores"/> or
- <Prp name="relpath" type="S" value="StoresChrome\Stores.tcStores"/>
depending on the browser the test has to run with.
This is not perfect, but the best for now.
1. executing complete nightly tests works perfectly fine
2. executing some tests locally might be a bit laborious
3. updating changed screenshots works ok (but 2. affects this)
not perfect, but worth a shot.