Seems unlikely.
In order to pre-check/validate ALL objects required within a test, they would have to be present.
Which isn't going to happen ..... without running all the tests. Many object won't appear without pre-work (ie - making a bunch of selections or entering some data or whatever) so I don't know how TC would validate them without actually running test.
Having just read the feature request, I see staff have similar concerns as to how you would actually do it.
And I REALLY don't like the idea that it deletes things it can't find. I can see that causing more problems than it solves! Maybe flag them for deletion I suppose, but no way I'd want it just trashing stuff by itself.
Any time I get an updated version, I generally accept it will need a minimum of two runs. One to catch up with updates, changes and new functionality. And then one to run and store test results. Except that invariably turns into (a minimum of) three runs. Run to find the changes. Run to test the updates. Run to run the tests.
I've long since got past the point where I ever expect ANY automated test (well, anything even vaguely complex anyway) to be totally change-proof. It's simply not realistic. If a dev changes a table name in a DB, and you run queries/tests against that table, your test will HAVE to be updated somewhere. It's unavoidable.
For me, it's more about keeping updates as simple and easy manage as possible. Not trying to avoid them completely as you never will.
Colin_McCrae It doesn't have to be ALL objects, it could be screenbased. Or on a start/stop base. (Tester presses starts to start the examination, and stops it, and after that TC does the compare.
See the comments on the mentioned feature request.
TC could make a match (screenbased) on the previously mapped objects/testscripts, and the current screen. This is very useful in Agile environments, in which the application changes every day, and the tester doesn't know where/what has changed.
Of course it's not up to me HOW to do it. (I'm not a programmer), it's just that with this feature request TC could increase market value for Agile / Devops environments significantly). In such environments there is no time for scripting/testscript mainentance: the intelligence should be built in the tool (TestComplete in this case) itself. It could be done because similar logic is already built into TC already, however it should be extended to the testrun phase. It's important for a testtool to have good bindings to the AUT. Like said in Agile the AUT changes frequently and TC should be changed to make it easier to work with UI changes in the AUT.
For me deletion is preffered: I like to have as much clean namemapping model as possible. Things which are not used please delete them. Could be different in different environments therefore this behaviour should be configurable (deletion/marked for deletion/disabled, whatever) in main configuration.
Changes are not all alike. UI changes can more easily be tackled by the tool (see feature request) then validation on DB values in the testscriptlogic itself. Thats where the scripter of the future will become more focused on. Detection of UI-Changes and handling them, need to be done by the intelligence of the tool.
Champion Level 3I don't want TestComplete doing anything to the name mapping automatically. If something has moved and others weren't made aware of it ahead of time, then I want the test to fail. The test can find holes in our process as well as our code.
If TC were to automatically remove mapped items and ignore them, wouldn't this invalidate your tests? What if a button is missing but should be there and test complete didn't report this?
cunderw Yeah. That was one of my thoughts behind auto-deletion causing more problems that it would solve. I mean, you just have to look at how automatic properties are captured out of the box. They almost always require some manual work on top to make them useful in the long term.
I guess in my case, most of the stuff I test (both desktop and web based) requires DB validation. And further validation on separate sites/applications. (Most are config/client type setups) For pure GUI testing, I guess I can see more potential advantages to it. But the point about triggers needing to be activated etc stands. Not easy to validate all the objects of anything but the most simple panel without interacting with it somehow.
Of course, you can always write your own routine to do basic "are all the bits there?" type validation. Based on your object maps. I have just such a routine but it's pretty slow as all it's doing is continuously scanning up and down the DOM looking for objects.
If this is how you are wanting to do this, then you should really be doing checks for if an object exists (using find child), if it does, do something with, if it doesn't log a message or warning, so you know to remove it from that section.
There really would be no good way for TC to intelligently update the name mapping if a button moves. It will have an entirely different parent and hierarchy and no way to know that for example Page1.Button1 is now actually Page2.Button1
Example pseudo code:
if(Aliases.Parent.FindChild("SomeProperty","SomeValue").Exists) {
//Do Something
}
else {
Log.Warning("UI Has Changed")
}
Using name mapping for all objects is not the approach one should really be using in this type of setting IMO.
The FindChild method works great for agile and frequently changing UI and would allow for exactly what you are wanting.
If a button moves on screen, but retains some of the same properties, FindChild could always find it without throwing an error.
If it's removed the sudo code above will not throw an error.
If it's moved to a new area of the application, you will have to update your test assets regardless of if the old mapping is deleted or not.
cunderw The button invokes a function. If the button isn't there that function cannot be tested (at least not via UI). In Agile just isn't time for complete specs so you just can't test against the specs. In that case you would just report "Didn't test that function because it wasn't there".
Of course the button could have moved to another screen also. TC would see that it is added in other screen and would indicate that it's "new" according to lastest namemapping model (yellow markings on screen - see feature request).
The tester would just update the script/keyword test to include the "new" button. This is a fast and efficient workable method in agile environments to keep up the speed at which the devs deliver in agile/continious builds/íntegration environments.
Next day would be another deliverance of the AUT and again this button could be placed somewhere else! It's crucial to automate as much as possible but to use extensive scripting as little as possible to keep up to speed!
cunderw In the end, TC creates the namemapping model, so it should be able to manage it as well. Maybe not enough in current version, then this should be redesigned. Otherwise TC will not be suitable enough in Agile/devops approaches to software development. To cope with fast deliveries of new software (updates), you have to have a great deal of intelligence in the tooling so the tester can update the framework acting more as a director, as a conductor in an orchestra.
Marsha_R Of course, it all depends the internal (within the business/organization) processes and procedures about what and when is communicated, along with the chosen software development approach (Agile, waterfall, incremental, hybrid).
That's why it should be configurable (let TC do changes automatically/suggest changes but let tester decide/do "nothing"), in TC's general settings.
Working in agile doesn't prevent the testers from knowing what they have to test... if you don't know what is new, there is probably something wrong in your processes
Champion Level 3Yes, Montikore That's exactly why I don't want TestComplete changing the name mapping without me knowing there was something wrong. Occasionally there are changes made to our part of the code base by another team and we aren't informed. I want my test to break so I see the change. It's a quick fix and I can also remind the other groups to share the information. It's a test for our processes as well as the code.
Marsha_R, Montikore It all depends on the environment (type of business, choosen approach for software development).
For example in Agile/CI and other short approaches with short development cycles the focus is primary on delivering working software, meeting customers requirements.
In such environments there is not always time to evaluate consequences of changes influencing other teams!
That's why the automation tools need to become better (more intelligent) in handling changes.
I would love to see TC handle name mapping changes automatically so I can focus on changes on higher level op abstraction in the short time I have for next delivery of the software!
Champion Level 3For another example, we have been using Agile for several years and we don't have a problem updating our tests if an issue is found.
Marsha_R Agile is a broad term and can be implemented in multiple forms and flavours, and degrees and in more/less delivery speed/frequency.
Some people say "we use agile" but in fact just use the name and really do not make fast deliveries of their software!
My point is: Smartbear (TestComplete) should be upgraded to support CI / CD and BDD processes, otherwise it will lose it popularity and marketshare. Times are changing and SmartBear has to "make the shift"!
Champion Level 3And testing is a broad term and can be implemented in multiple ways. Some people say "we know the best way" and really all they know is their own situation.
My point is that TestComplete is a good generic tool now and what SmartBear should do is what they think is best to improve it.
Marsha_R Of course it's up to SmartBear internal teams/ product owner to decide in what ways to evolve the product(s).
But we, testers, QA-engineers, developers, test managers, etc, are the eyes and ears of the product. We know best what is happening in daily work environments. And that's why we need to discuss things on forums. Eventually SmartBear picks up these signals and do with it they think is best for product evolution!
