Forum Discussion

ctira's avatar
ctira
Occasional Contributor
2 years ago

Inconsistent test duration

Hello.

 

I'm doing some UI testing with TC and can't really explain why I have the same test (same environment, parameters, everything) run with very different durations: fast run everything is done in 7-8 minutes and slow runs are don in 40-45 minutes. From the best that I can notice, the fast runs seam to have the objects that the functions search for (for clicking, typing text, etc.) somehow cached. It's like manually in the fast run you can't keep up with how the test is performing each particular actions (click, type) and with the slow run you can't figure out why it hasn't found the object it was searching for yet.

Again, I'm not doing anything between consecutive runs and today I got the following situation: did 4 runs of the same test, runs 1,2 and 4 were fast runs and run 3 was a slow run.

 

Has anyone seen something similar?

 

Thanks for your help!

Object Recognition
web testing

14 Replies

Sort By
  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor
    2 years ago

    I'm assuming this is a web application?

    Basically, you nailed it with object recognition.  When TestComplete needs to find an object and click on it, sometimes the object may not be available right away so it delays.  I'm assuming there are no errors in your project which means TestComplete eventually finds the object but it's that "find" time that sucks up your run time.  

    Something to take a look at is whether or not you have "extended find" turned on in some of your mapped objects.  IF an object is not found right away, sometimes "extended find" can trigger longer runs as this triggers a scan of the full object tree.

    My suggestion: 

    1) Rework your test to include some better "wait" code to wait for components to load before you proceed with the test.  While this may slow your "fast" runs down a little bit, it WILL make your code more consistent in timing by building in checks.  If you think about it, as a human, you respond to small triggers and stimuli as you're working with an application that indicate that it's "ready" for you to interact.  You should code automations likewise

    2) Rework your name mapping to remove extended find where it's not necessary or useful.  It has it's purpose in some web applications with a lot of <DIV> elements but it can slow things down.

    • ctira's avatar
      ctira
      Occasional Contributor
      2 years ago

      This is a web application.

       

      With both slow and fast runs I get a green log for my test with no issues (problem is the big time differences of the different runs)

      1. We already have the function that clicks on an object wait until the object can be found (or more than 20 seconds have passed from the beginning of the search). Including this wait was already present with both slow and fast runs and so the 7min fast run is already waiting for the objects to become visible.

      2. We don't use name mapping in the UI part of the web application because of the nature of the pages that we work with. I disabled the "Extended find" option but I haven't seen any difference with and without it active (still have slow or fast runs kind of randomly)

       

      Kind regards,

      Cristi

  • rraghvani's avatar
    rraghvani
    Champion Level 3
    2 years ago

    What version of TestComplete are you using? Are you using scripts to perform your tests? Are you performing any sort of refresh to update the tree of currently running processes?

  • ctira's avatar
    ctira
    Occasional Contributor
    2 years ago

    I'm using TC version 15.45.

    The tests are done with using scripts.

    I'm not refreshing or updating the tree during the test runs.

  • rraghvani's avatar
    rraghvani
    Champion Level 3
    2 years ago

    What scripting language are you using? How are you searching for objects? Are your mapped objects correctly structured, i.e. having parent and child relationships. Do disable Extended Search. Have you checked there's no issues with your web application?

     

    • ctira's avatar
      ctira
      Occasional Contributor
      2 years ago

      I'm using javascript. For searching the objects during the test we use webpage.FindChildByXpath(xPath). In the Object Tree I can properly see parent/child relationships.

      Like in my previous reply, I did disable Extended Search but didn't see any differences.

      What kind of issues for the web app do you expect? Everything behaves as it should, both slow and fast runs of the test get a green log. Only difference is (from what I can tell) that the fast run doesn't seem to need to search for the objects as it already knows where they are and where to click/type test, whereas the slow run needs to search for them.

      • AlexKaras's avatar
        AlexKaras
        Champion Level 3
        2 years ago

        Hi,

         

        Try to take a closer look at test log - it contains Time Difference column. Check if unexpectedly long time is spent for some actions. Maybe this will provide you with some ideas.

         

  • rraghvani's avatar
    rraghvani
    Champion Level 3
    2 years ago

    From ~7 to ~40 minutes, that a big difference!

     

    I suggest you work on a small area of your web application and debug the code. You will need to repeat this a few times, and visually analyse what's going on. Hopefully, you will see some patterns merging that will give you some clues

    • ctira's avatar
      ctira
      Occasional Contributor
      2 years ago

      I've managed to see a small pattern but it doesn't explain everything (yet).

       

      Whenever I had a fast run the browser was still opened from a previous run. Maybe that helped out with not getting the cache deleted. When I've disabled the "Extended find" I didn't have any fast runs.

      The problem that I'm struggling to understand now is why there are still differences when the browser remains opened.

      Summing it up:

      - browser is closed at the beginning of the test and needs to get opened - always a slow run

      - browser is opened when the test begins (we have a check to open the browser only if it's not opened already) - sometimes a fast sometimes a slow run

      - with 'Extended find' disabled I haven't gotten a fast run

      - with 'Extended find' enabled I do get either a slow or a fast run

       

      I've did several tests considering smaller parts of the test and what I saw was that in slower runs every action takes a couple of seconds to perform (they actually search for the required objects), whereas in the fast runs like I said before, they find the objects instantly.

       

      Kind regards,

      Cristi

  • eykxas's avatar
    eykxas
    Regular Contributor
    2 years ago

    Maybe the browser cache is responsible for this behavior ?

  • rraghvani's avatar
    rraghvani
    Champion Level 3
    2 years ago

    It could be a number of things, but it's difficult for us to provide a solution. We can only provide suggestions. You will have to try your best to diagnose the issue.

     

     

     

  • ctira's avatar
    ctira
    Occasional Contributor
    2 years ago

    Hi guys,

     

    I managed to get consistent fast runs in the end. Since I saw that fast was really an option, I started do dig a little bit in our framework and found out what slowed us down. Basically with our click function we were waiting for the needed object and that performed a search by xpath, then we checked if the object is visible again by its xpath (that performed again a search for the same object) and finally we clicked on that object by its xpath (yes, you guessed it - by searching for it again). In then end clicking on any object performed several unnecessary additional searches that slowed us down. I refactored this part and now get consistent 7-8 minutes runs and no longer the 40+ minutes needed...

     

    Still I have not been able to figure out why sometimes things were running fast... I am still assuming that it had something to do with caching but why it was so inconsistent I have no idea.

     

    Thanks for all your suggestions.

     

    Cristi