As we continue to add features in TestComplete that are valuable to you, we would like your feedback on a particular capability that we’re considering to implement in the near future.
We’re thinking about making TestComplete and TestExecute accessible via a REST API, enabling you to access various artifacts with TestComplete remotely with ease. A few questions:
Your feedback is precious to us.
Sounds like a very interesting TestComplete feature!
Sounds very interesting!
As for me, I would find useful, if I can get some staticstics about currently running tests. E.g. test queue, tests failed, checkpoints failed for particular tests, screenshots... And definitely I should be able to start / stop / pause / restart tests via api calls.
AFAIK, REST API already exists in TestComplete/TestExecute and is used to drive TE from within TestLeft. It is just disabled in TC by default but it is no problem to enable and consume it (https://community.smartbear.com/t5/TestComplete-General-Discussions/How-to-provide-an-object-from-th...). Please correct me if I am missing something for the REST API been suggested.
On the other hand, it is OS requirement to have active not locked user session in order to be able to run UI tests. Unless it is possible to check for the active user session existence and provide it if necessary via suggested REST API, I hardly see the benefit of REST over the regular command line. (There is no problem to remotely login into test box and start test within remote desktop session using command line tools now if the account has proper permissions.) I guess, that REST API might be more convenient if tests are executed within a cloud (e.g. AWS), but not sure at the moment about licensing.
I would agree with @baxatob that it would be convenient to be able to get information about remote active test run without the necessity to attach to the remote session. (Attaching itself is not a problem, but it is not convenient to switch remote session back to console.)
However, I think that it may be even more useful and convenient if TestComplete provides some listener component that can be installed standalone. For example, on the Jenkins or some other CI server. Then, when needed, running remote instances of TestComplete/TestExecute can use something like push notifications to feed this listener with the info defined by test developer. (I am not sure that it is a good idea to push all test log to the listener. As a possible option, Indicator object can be enchanced with additional parameter to push its contents to the listener when pushing functionality is enabled for the test.)
This listener component can be used than to provide an overview of the active test runs (grouped by test machines) to console, CI server panel, messaging service, etc.
I second @AlexKaras and @baxatob . It seems, for me, that the best use of a REST API for TC/TE would be to have some sort of remote dashboard where I can get a quick view as to the status of a current test run. Perhaps some sort of API query that could return current test/script/keyword test being executed, maybe a glimpse at the last few log messages, error counts, etc. I can see this being utilized through CI programs like Jenkins, QAComplete, or other such things that would allow a "real-time" visibility into what is currently being executed.
This would be fantastic. I would love to be able to see current project, test item, statuses etc... Also, being able to add things to projects before running, such as script units would be nice, as well the ability to enable and disable test items prior to run projects.
for me It will be a nice to have but not exciting .
I would rather excited it TC have editor capacity such as visual sudio or Eclips
Or improved reporting capacity, I know QA complete has reporting capacity but there are people who do not need to have fullyfledge test management suite.
We are using Jenkins to run our tests, When we say REST API I believe Jenkins plug-in also needs an update.
Things that can be helpful when we have REST API for TestComplete
Other then REST API I really like to see improvements on Code Editor like other Ecplise/Visual Studio
Thank you all for the great feedback! Your assistance is very appreciated, and all your responses are essential for us.
@AlexKaras, you are right, we already have a REST API for TestExecute. You are also right when pointing that this API was created for internal communication between TestExecute and TestLeft. My question was about a public API that can be used by external clients to work with TestComplete/TestExecute.
We started down the path of ReadyAPI but have just since decide to posibly pulled the plug on it.
Since we are use TC with the ReadyAPI plug-in, It would have been nice to have better stats capabilities, access to send dynamic data instead of the static Global/System parameters, also better logic handling on Assertions(if/else).
We have decide to use TC exclusively to generate dynamic REST calls and handle all checkpoints and stats. This is more inline with the rest of our UI interfaces and WebServices.