automate the validation of testcomplete tool

Question

Hi all,&nbsp;My requirement is to validate testcomplete tool&nbsp;itself.How do I automate the validation of testcomplete tool&nbsp;?&nbsp;Thank you.

tristaanogre · Accepted Answer

Just to note how we solved the three requirements that I indicated:
&nbsp;
&nbsp;
1) We determined to use a data driven framework based in JScript that used CSV data to generate the tests. Within the CSV data for each test executed, we provided space for the tester to indicate the specific requirement ID being validated by the indicated test
2) Within that framework, we added an additional column for verbose description of what each test step included so that a non-technical person could read the output log and understand the sequence of the tests
3) Built in to TestComplete are values to return the machine name, the operating system, and the logged on user name. These were included as a log entry at the start of every test execution to be able to report the environment for the test
4) We included in the test run an export of the test log to an MHT file format to a shared directory so that the test evidence was "portable" to anyone who needed to be able to read and verify the test completion.With these 4 things, we were able to meet the three requirements that I mentioned. There were more requirements than those three but I don't recall them in detail at this time but we implemented similar demonstrations to show that TestComplete could provide accurate testing and sufficient test reports to meet our CFR Part 11 compliant processes and procedures.If you think this is extreme, we had to do it for TFS, for Visual Studio, for MSSQL Server, etc., to prove, for each of these, that the tool met our process and procedures to maintain our CFR Part 11 compliance. Some of the stuff we documented as "widely used industry standard COTS application" so we didn't have to test and prove EVERYTHING, just those things that may be atypical according to&nbsp;our particular use.

tristaanogre · Answer

A couple of years ago, I did a validation process for TestComplete for a company I was working for at the time. &nbsp;The general idea was to do basic tests and approve the application as a Commercial Off The Shelf (COTS) tool for use for software testing in a FDA Part 11 CFR compliant environment.
&nbsp;
I worked directly with the SmartBear folks at the time. Generally speaking, we needed a sign off from SmartBear on their testing techniques of their various software features. &nbsp;On my side, we needed to first of all document the specific use of the tool to make sure we clearly defined the parameters under which it would be used. We then went through and built some basic tests, not of our software, but generally of the tool to prove that the tool could be used under those parameters complete with screenshots, printouts, and documentation of the tests. This was then signed off on by me (the validator), our CFR compliance officer, and the testing manager.As for automating it... well, the only "automation" in that process was to build a set of validation tests to prove the CFR compliance and that it would fit our required parameters.

tristaanogre · Answer

As an organization, we spent time discussing what the TestComplete tool would fulfill in the way of a role in our testing structure. Being a Part 11 Compliant environment, every tool must be evaluated but TestComplete can do a LOT of things, not all of which are necessarily within scope of the use within our environment. So, we set up the boundaries. &nbsp;We made a list of things it would be used for, how we would achieve that, what aspects of TestComplete we would be using to do those things, etc. Essentially, we limited the scope of our use. It made me a bit sad, to be true, because there were a lot of things that I thought we could use TestComplete for but, within the strict FDA requirements, we couldn't do so. &nbsp;
&nbsp;
Once the scope was defined, we then set up a project within TestComplete that would demonstrate how TestComplete would achieve those parts of the scope. We built tests to show logging, generating reports from the logs, exercising tests against our particular development platform (SharePoint), and showing that the tests could report who, specifically, ran the tests. &nbsp;I then executed those tests, collected all the evidence from those tests that demonstrated compliance to the scope, and bundled them into a set of documents that I took to our compliance officer. &nbsp;He reviewed the results and signed off on it.We SPECIFICALLY validated a particular version of TestComplete. Again, this was part of the Part 11 compliance protocols. If we determined that we would upgrade TestComplete to a newer version, we'd have to repeat the tests to show that the tool was still within scope of the COTS requirements.
&nbsp;
Hopefully this helps. &nbsp;If you have any more questions, feel free to ask.

tristaanogre · Answer

Those limitations were solely within our organization and what we had determined within our documented testing processes and procedures as acceptable use of the tool. A different organization that is FDA Part 11 compliant may have different limitations because their processes may have different requirements. &nbsp;As for the summation you gave, that's a bit more specific. For the validation test, we didn't actually test our software but we created a "generic" project in TestComplete that demonstrated those features we intended on using and their ability to fulfill the requirements we had set aside for our COTS validation. &nbsp;So, essentially, we documented something like:1) Needs to be able to provide output that maps executed tests to documented requirements
2) Needs to be able to provide detailed logs indicating the steps executed for those tests
3) Needs to be secured to document the workstation environment and logged in user to trace the execution of the tests to the person/environment in which it was executed.
&nbsp;
This is just a minor example of what we documented. &nbsp;We then set out to build a set of tests that would show that, yes, we can do precisely that... that TestComplete had the ability to meet those indicated requirements. &nbsp;Once we validated that, we could then do whatever we wanted with TestComplete to actually automate our tests within the established scope and parameters under which we validated the tool.

marsha_r · Answer

What is it about TestComplete that are you trying to validate?

jlyon · Answer

I am trying to validate its important features like functional testing, regression testing, data-driven testing,distributed testing etc.Basically trying to validate its intended uses.

Forum Discussion

automate the validation of testcomplete tool

12 Replies

Related Content

TestComplete Web Automation Engineer

How to Automate Tests in TestComplete

Automation Script OAuth 2.0 Azure not fully validated

Scheduling automated TestComplete Script on TestExecute machine

TestComplete with Zephyr Scale

Recent Discussions

Floating window bleep overlapping window takes longer time (~10 min) to click the button on it.

Test case randomly fails with "parent object was not found"

Does TC have a .within() method similar to Cypress.io?

SessionCreator /Timeout option not firing EventControl_OnTimeout

testcomplete feature Intelligent Quality Add Ons