Inconsistent test duration

[ctira]

Hello.

I'm doing some UI testing with TC and can't really explain why I have the same test (same environment, parameters, everything) run with very different durations: fast run everything is done in 7-8 minutes and slow runs are don in 40-45 minutes. From the best that I can notice, the fast runs seam to have the objects that the functions search for (for clicking, typing text, etc.) somehow cached. It's like manually in the fast run you can't keep up with how the test is performing each particular actions (click, type) and with the slow run you can't figure out why it hasn't found the object it was searching for yet.

Again, I'm not doing anything between consecutive runs and today I got the following situation: did 4 runs of the same test, runs 1,2 and 4 were fast runs and run 3 was a slow run.

Has anyone seen something similar?

Thanks for your help!

[rraghvani]

What version of TestComplete are you using? Are you using scripts to perform your tests? Are you performing any sort of refresh to update the tree of currently running processes?

[ctira]

I'm using TC version 15.45.

The tests are done with using scripts.

I'm not refreshing or updating the tree during the test runs.

[tristaanogre]

I'm assuming this is a web application?

Basically, you nailed it with object recognition.  When TestComplete needs to find an object and click on it, sometimes the object may not be available right away so it delays.  I'm assuming there are no errors in your project which means TestComplete eventually finds the object but it's that "find" time that sucks up your run time.  

Something to take a look at is whether or not you have "extended find" turned on in some of your mapped objects.  IF an object is not found right away, sometimes "extended find" can trigger longer runs as this triggers a scan of the full object tree.

My suggestion: 

1) Rework your test to include some better "wait" code to wait for components to load before you proceed with the test.  While this may slow your "fast" runs down a little bit, it WILL make your code more consistent in timing by building in checks.  If you think about it, as a human, you respond to small triggers and stimuli as you're working with an application that indicate that it's "ready" for you to interact.  You should code automations likewise

2) Rework your name mapping to remove extended find where it's not necessary or useful.  It has it's purpose in some web applications with a lot of <DIV> elements but it can slow things down.

[rraghvani]

What scripting language are you using? How are you searching for objects? Are your mapped objects correctly structured, i.e. having parent and child relationships. Do disable Extended Search. Have you checked there's no issues with your web application?

[ctira]

This is a web application.

With both slow and fast runs I get a green log for my test with no issues (problem is the big time differences of the different runs)

1. We already have the function that clicks on an object wait until the object can be found (or more than 20 seconds have passed from the beginning of the search). Including this wait was already present with both slow and fast runs and so the 7min fast run is already waiting for the objects to become visible.

2. We don't use name mapping in the UI part of the web application because of the nature of the pages that we work with. I disabled the "Extended find" option but I haven't seen any difference with and without it active (still have slow or fast runs kind of randomly)

Kind regards,

Cristi

[ctira]

I'm using javascript. For searching the objects during the test we use webpage.FindChildByXpath(xPath). In the Object Tree I can properly see parent/child relationships.

Like in my previous reply, I did disable Extended Search but didn't see any differences.

What kind of issues for the web app do you expect? Everything behaves as it should, both slow and fast runs of the test get a green log. Only difference is (from what I can tell) that the fast run doesn't seem to need to search for the objects as it already knows where they are and where to click/type test, whereas the slow run needs to search for them.

[AlexKaras]

Hi,

Try to take a closer look at test log - it contains Time Difference column. Check if unexpectedly long time is spent for some actions. Maybe this will provide you with some ideas.

[ctira]

Hi Alex,

All of the actions take a few seconds longer in the slow version of the test run (not something extraordinary - a couple of seconds added for each step) but it all adds up.

Cristi

[rraghvani]

From ~7 to ~40 minutes, that a big difference!

I suggest you work on a small area of your web application and debug the code. You will need to repeat this a few times, and visually analyse what's going on. Hopefully, you will see some patterns merging that will give you some clues