Does anyone has any experience in building new framework to test UI application using python

[MaheshBhoite]

We are planning to use TestComplete to automate UI regression, does anyone has experience in building framework using Python as scripting language?

I have searched internet for some basic framework for so long but couldnt find any.

[baxatob]

Hi,

We are using python for coding inside of our framework. It works fine and relatively fast especially with the latest version of TestComplete. 

[MaheshBhoite]

Hi there,

Thanks for your prompt reply,

may I ask what scripting language your framework is written in?

[baxatob]

As I said, we are using Python.

[MaheshBhoite]

Thanks,

What framework are you using? I mean keyword driven, data driven etc.

Can you please give us some starting points on how you build that framework?

With very little information available over internet, not sure how to start building it.

[baxatob]

We are using all of mentioned features. Different tests require different approaches. Only you know what are you going to test. It can be a server application, or may be iPhone app, or some desktop software. Providing us with more details, you will increase a chance to find a roadmap here 🙂

And about the starting point. TestComplete is perfectly documented, and I will suggest to start from here: https://support.smartbear.com/viewarticle/71476/

[AlexKaras]

Hi Mahesh,

The problem is that the word 'Framework' is extremely overloaded and means significantly different things when used by different people and, according to my experience, it means nothing unless provided with the description of what does it mean for every given context.

So, first and foremost, you must create your definition of 'framework' and only then others will be able to help you.

According to http://www.oxforddictionaries.com/definition/english/framework, framework is "an essential supporting (underlying) structure [...]'. And because the support (which in our given contents can be also understood as a requirement) can be provided at different levels (from some technical low-level up to descriptive conceptual high-level one), you can end-up talking about different frameworks.

For example, you may require that your (automated) tests do not use hardcoded test data but only data provided by the external files. This will lead you to creation of some data structure and the code that will read and, maybe, also write data from and to this data structure and exchange test data with your tests. Most probably, you will document how to work with all this stuff so that your data structure is not broken and require from all relevant people to follow the described rules. As a result you will get a framework. Whether or not will you use for this Data Driven functionality of TestComplete is another question. (Though it may be specified in your requirements and thus be included or not into your framework.)

You may also require that all high-level actions that can be done within your tested application (application start, menu item selection, actions with grids, etc.) must be described using defined words (provided with the set of these words and their exact meaning). In this case you will get another framework. Whether or not you will use the Keyword functionality of TestComplete to support it on a technical level is again, another question.

You can combine the above two requirements and, as a result, you will get a third framework, specific to your organization, tested application, staff skills, etc.

On the lower level, you may require, for example, that all actions against UI controls of your tested application (selection, checking, data entering, etc.) must be provided via wrapping functions only but never by calling .Select(), .Keys() and other methods provided by TestComplete. In this case you also will get a framework. (Though the more correct name for it, from my point of view, is 'api' or 'library functions').

Summarizing: start from defining what will be required from your tests. Then decide how these requirements can be implemented. Then consider whether the implementation will use some functionality provided by TestComplete or you will create it yourself.

I believe that it is quite obvious from the said above that you may get some replies / find some information if you ask about low-level implementation (low-level framework / library organization). But the chances are pretty low that somebody will provide you with the description of their high-level requirements (high-level framework). These chances are low just because the high-level frameworks are usually pretty specific to the given organization and project and thus are irrelevant to you. (You may get a generic description like "we are using Jira to manage requirements and integrate it with QAComplete for test plans and test results management. We are using TestComplete for tests automation and our tests do not contain test data inside it but use external test data stored in SQL database"... but this will be just a generic description.)

[MaheshBhoite]

Thanks Alex, this is wonderful explanation on framework and what it would mean for different people with different perspective.

What I am looking for is, we have a desktop application written in C#, and we want to automate the regression tests that are currently being executed manually.

We are asked to use Python as scripting language because that's the vision going forward in our organization. Now we have to develop a TestComplete project to read test cases and test data from excel file and execute it on the application.

I could not find any sample project written in Python in TestComplete (even the TestComplete samples provided are in C#,Delphi,JScript and VBScript), how should I begin to write this project?

We want to perform below actions.

1) Open excel file and test cases and data

2) Launch the tested app

3) Perform actions coded in script

4) Error handling/exception handling/ events in TestComplete

5) Report the test execution status via email

[AlexKaras]

Hi,

> We are asked to use Python as scripting language [...]

Just a quick note related to Python that personally I got a big problem with. Hopefully, others will comment what I did wrong or how they solve this problem:

The case is that Python does not have any special means (parenthesis, curly braces, etc.) to group script flow instructions (loops, conditions, etc.) and the only means are indents. If, for example, you created initial file in the editor that do not use tabs for indents but only spaces, do later some modifications in the editor that uses tabs for indents and then open the file in the editor that expands tabs to spaces (or treat tabs to equal not to 8 spaces, but to some other number of them), the chances are high that this will break initial indentation of the code and thus can break all code logic (because some lines of code can move in or out of the code flow control statements).

Restore of the initial code logic requires significant efforts because there is no other means except (broken) indentation that defines whether or not the given line of code must be within or outside, say, a loop statement. So, in order to restore the position of the line of code (inside or outside of the flow control operator) one must to get the complete understanding of what the given piece of code was expected to do initially.

Hope, that this my negative experience will be clarified by those who have positive experience with Python.

[baxatob]

Hm... I never have had the problem mentioned by @AlexKaras.

I have set up the built in code editor of TestComplete in the same way as I have set up all of my other code editors and IDEs (like IDLE, Eclipse and Notepad++). Actually we only need to define the number of indents (spaces) per one block of code.

Also I am always using the source control system to store and update my code in a safe way. It's a kind of insurance for the case if I have broken something anyhow.

=====

@MaheshBhoite, first of all you should define your framework architecture. In a very common approach you need to separate your test-steps from the test data, from application mapping and logic and from configuration and infrastructure. In other words your whole test should looks like:

myApp.start()

myApp.login()

myApp.doSomething()

myApp.validateSomething()

myApp.logout()

myApp.stop()

And nothing else.

Definition of all methods like start(), login() etc. should be hidden from your test in the separate module.
All infrastructure configuration like server name, data bases etc. should be hidden from your test in the separate module or inside of config- or ini- files.
All test data like credentials and other input values should be hidden from your test in the separate module, in variables or somewhere else.
All elements mapping should be hidden from your test in the separate module, classes or/and in the NameMapping repository.

...and so on