Forum Discussion

pretzelz's avatar
pretzelz
Occasional Contributor
13 years ago

TestComplete fails to perform consistently or reliably.

Hello TestComplete community.



I've been trialling the latest version of TC for the last 3 weeks, and have become very frustrated. It seems most of my time is spent trying to fix old previously successful code that TC generates, rather than use TC as a tool to move forward with what I need to accomplish.



Issue 1:

=====

The most frustrating (and simply put) issue I'm having is that I can record (not LLP) the code for clicking a button on the software I am testing. When playing back, this works perfectly. I continue with my project. (This button remains static on the user interface). Later, I run the same code, possibly after a restart of TC - and the previously successful code fails to work. So I again record this action - and sometime later, this code again fails to work. On this particular issue, I have got a side by side comparison of working captured code and not working captured code. There is a difference in both, yet the action carried out is the same. I have recorded these particular examples on a non-changing part of interface involving just one click on a specific button.




//does work (new code)



Aliases.EXEFramework.HwndSource_LayoutManagerWindow.LayoutManagerWindow.Grid.Viewbox.ModuleBar.StackPanel.LitRadioButton3.Viewbox.Canvas.Path.Click(26, 11);



//doesn't work (old code)


Aliases.EXEFramework.HwndSource_LayoutManagerWindow.LayoutManagerWindow.Grid.Viewbox.ModuleBar.StackPanel.LitRadioButton2.Viewbox.Canvas.Path.Click(13, 18);



As you may have spotted here, one reference is different:  LitRadioButton2    and LitRadioButton3.           If nothing changes in the interface, why would it record different names at different times - and then later fail to recognise the first name?





Issue 2

=====

In a recorded sequence of movements (about 15 seconds of 25 movements around the interface, involving dragging and selecting) when playing back this sequence (not LLP recorded) all can be fine. When playing it back on another occasion, it can skip a beat at a random point in the process and not carry out an action correctly, or for example, the mouse will miss an object when going for it.)





Issue 3

=====

Same as issue 2, only mouse references can go off course at different points within recorded LLP sequences. When re-recording them to make them aligned again, they can go off course at a later time.







I am really at the point of trialling other automation software, because TC is not reliably capturing and performing consistently to allow me to move on. I would be grateful to hear from anyone else who has experience similar issues when using TC and if you came across any particular solutions.



Many thanks.



System spec:

Xeon E5520 @ 2.27 Ghz

RAM 6GB

Win7 64bit



(similar issues on an old P4 machine)









  • pretzelz's avatar
    pretzelz
    Occasional Contributor
    13 years ago
    Well, it seems I've solved a few mysteries over the past day. This is the most notably point I have found: when recording the macros, I need to wait a good amount of time (no less than about 6 seconds) at each step within my navigation during recording. This seems to be the best time to allow TC to talk with our software and gather all the code it requries to execute successfully. I suspect it may also be something to do with waiting for the hover over-caption that appears when hoving over a particular button. Perhaps only after the caption has disappeared does our software send out the full information about what is taking place. I hope this tip helps others in a similar sitution.



    David

  • sastowe's avatar
    sastowe
    Super Contributor
    13 years ago
    This is MY personal opinion, not the opinion of Smart Bear and is not intended as a criticism of Smart Bear. Over the years, I have been exposed to several test automation tools. In most, if not all, cases, they promise record and playback of tests. This is so completely nonfunctional as to be a myth, in my opinion. One reason is what you see. The object that you are clicking on is not immediately recognized by Test Complete. Therefore coordinates on the closest known object are clicked. But computers vary, resolutions vary. How applications handle window and screen element positions vary.



    In my opinion, test automation is a software development venture as much as the development of the software that is being tested. Education of the chosen tool is necessary. (I love Test Complete, by the way. It is not perfect, and I have not used any other tools in a few years. But for me, this works great.) Design the tests, reusable components to save time, how you want to examine output....



    In my opinion, one should not buy the promise of open the box, record and playback tests. If one is to do test automation, one ought to spend the time and money to formulate a strategy and plan. YMMV.





  • pretzelz's avatar
    pretzelz
    Occasional Contributor
    13 years ago
    I would like to ask this question of TestComplete...   why can I record the following:  



    1. click button (reveals menu).

    2. select option from menu.





    ...yet it hangs when playing it back?  



    1. clicks the button... waits and the menu doesn't pop up



    (Interestingly, TC works in other places within the software with a similar style button and submenu selection)





  • pretzelz's avatar
    pretzelz
    Occasional Contributor
    13 years ago
    (and thank you for your post Stephanie - your insight was appreciated)
  • rachel_dunlop's avatar
    rachel_dunlop
    Occasional Contributor
    13 years ago
    Thanks for posting your solution David, because it just helped me with a very very similar issue! By adding a delay before the button click my test now passes perfectly!
  • rickm's avatar
    rickm
    Contributor
    13 years ago
    With regards to the LitRadioButton2 vs LitRadioButton3 in the path, did you by any chance have more than one instance of the application running during recording or playback? Perhaps minimized?



    I see similar things when I'm including the recording of interacting with Windows Explorer (file manager). If I have another instance of Explorer running when the test is played back, I see similar path changes and the test fails.