Forum Discussion

leeasdf9's avatar
leeasdf9
Frequent Contributor
11 years ago

Possible to run an execution list of Testcomplete Project with Test Execute?

Dear Tanya, Dear Support Team,

I just get started with testexecute 9 and would like to run the test complete projects with testexecute on our VMs.

I did read article "Run an individual test item via command line", i think it should be also possible to select and run an execution list from my testcomplete projectsuite via command line or(pls. see sample list below)?

Could you please help me to create a batch file for this task? Thanks for the support.

Br,

Lee



Projectsuite

 testproject1

   keywordtest1

   keywordtest2

   keywordtest3

 testproject2

   keywordtest1

   keywordtest2

   keywordtest3

...

 
  • Philip_Baird's avatar
    Philip_Baird
    11 years ago

    Hi, another option, which I have just implemented, is to wrap TestExecute as a COM object in a C# Class Library which also contains an XML file reader.


     


    I can feed it an XML file such as the following


     


    <?xml version="1.0" encoding="utf-8"?>


    <tests>


    <test type="ProjectItem">


    <projectName>ProjectB</projectName>


    <projectItemName>ODT</projectItemName>


    </test>


    <test type="Project">


    <projectName>ProjectB</projectName>


    </test>


    <test type="ProjectTestItem">


    <projectName>ProjectA</projectName>


    <projectTestItem>ProjectTestItem3</projectTestItem>


    </test>


    <test type="RoutineEx">


    <projectName>ProjectB</projectName>


    <unitName>Unit2</unitName>


    <routineName>testWithParameter</routineName>


    <parameters signature="source,destination">


    <parameter name="source">thesource</parameter>


    <parameter name="destination">thedestination</parameter>


    </parameters>


    </test>


    <test type="Routine">


    <projectName>ProjectA</projectName>


    <unitName>Unit2</unitName>


    <routineName>testUnit2</routineName>


    </test>


    <test type="Suite">


    <projectName></projectName>


    </test>


    <test type="TestByName">


    <projectName>ProjectB</projectName>


    <testName>KeywordTests|KeywordTestUnit</testName>


    </test>


    </tests>


     


    and the XML file reader reads this file and uses a TestExecute COM object to execute each test based on the attributes and elements of each <test>.


     


    It took me ~1.5 days to write something that works, I just have a few Exception scenarios to code in so your friendly neighbourhood developer should be able to whip something up pretty quick.


     


    For mine, this is a lot nicer and cleaner than working with batch files and the TestExecute command line limitations.


     


    Regards,


    Phil Baird

  • Philip_Baird's avatar
    Philip_Baird
    Community Expert
    11 years ago

    Hi, another option, which I have just implemented, is to wrap TestExecute as a COM object in a C# Class Library which also contains an XML file reader.


     


    I can feed it an XML file such as the following


     


    <?xml version="1.0" encoding="utf-8"?>


    <tests>


    <test type="ProjectItem">


    <projectName>ProjectB</projectName>


    <projectItemName>ODT</projectItemName>


    </test>


    <test type="Project">


    <projectName>ProjectB</projectName>


    </test>


    <test type="ProjectTestItem">


    <projectName>ProjectA</projectName>


    <projectTestItem>ProjectTestItem3</projectTestItem>


    </test>


    <test type="RoutineEx">


    <projectName>ProjectB</projectName>


    <unitName>Unit2</unitName>


    <routineName>testWithParameter</routineName>


    <parameters signature="source,destination">


    <parameter name="source">thesource</parameter>


    <parameter name="destination">thedestination</parameter>


    </parameters>


    </test>


    <test type="Routine">


    <projectName>ProjectA</projectName>


    <unitName>Unit2</unitName>


    <routineName>testUnit2</routineName>


    </test>


    <test type="Suite">


    <projectName></projectName>


    </test>


    <test type="TestByName">


    <projectName>ProjectB</projectName>


    <testName>KeywordTests|KeywordTestUnit</testName>


    </test>


    </tests>


     


    and the XML file reader reads this file and uses a TestExecute COM object to execute each test based on the attributes and elements of each <test>.


     


    It took me ~1.5 days to write something that works, I just have a few Exception scenarios to code in so your friendly neighbourhood developer should be able to whip something up pretty quick.


     


    For mine, this is a lot nicer and cleaner than working with batch files and the TestExecute command line limitations.


     


    Regards,


    Phil Baird

  • nnguyen_1's avatar
    nnguyen_1
    Contributor
    11 years ago








    lee hou

    Hi Robert,

    Thanks for your reply, sorry about confusion of "execution list", i mean list of testitems of the project editor, please also see also attached snapshot.

    As you see i have different "execution lists" for different test tasks. i would like to be able to run any of the list with testexecute.

    And for each test there is predefined timeout limit, i created timeout event handler if a testitem runs timeout it should be stopped and go to next testitem.

    Ifi can neither define timeout limit of testitems nor run the list in testexecute, my test performance will suffer from testrun without timeout handler.

    Could you help me to find  a solution? thanks for the support.

    Br,

    Lee




     


    Post attachments:


    executionlist.png

    @lee hou: I think these are what you need:

    1. VBScript&colon; http://support.smartbear.com/viewarticle/54654/. This one works perfectly for me now. 

    2. JavaScript&colon; http://www.sqaforums.com/forums/automatedqa-smartbear-testcomplete/148259-unable-run-tests-under-batch-group.html



    Example: 

    cscript "<path>\RunTestItemGroup.vbs" "<projectfile>.pjs" /p:<projectname> /testitemgroup:<groupname> /r /silentmode /e



    Only one limitation is that if you want to execute a group by TestExecute command, you need to make sure your group is checked off in the project file. This one is inconvenient when you want to debug a particular group only in your development time. You need to disable unwanted groups before debugging. However, if your groups don't cross each other, there is no problem at all. 
  • TanyaYatskovska's avatar
    TanyaYatskovska
    SmartBear Alumni (Retired)
    11 years ago

    Hi Lee,


     


    What if you run the whole project suite like this?


    "C:\Program Files\SmartBear\TestExecute 10\Bin\TestExecute.exe" "C:\Work\My Projects\MySuite.pjs" /r /e


     


    In this case, TestExecute will execute the test in the following way:


    Project Suite


     - Project 1 


      - TestItem1


      - TestItem2


      - Group 1


        - TestItem11


        - TestItem12


      - <All enabled test items>


    - Project 2


     - TestItem1


     - TestItem2


     - <All enabled projects>


     


    Note: you need to enable/disable items/projects in TestComplete before executing the test in TestExecute.


     

  • Philip_Baird's avatar
    Philip_Baird
    Community Expert
    11 years ago

    Hi lee, perhaps I need to clarify, you can't just give the XML file I posted to a TestExecute COM object.


     


    The Class library needs to follow the following basic flow:



    1. Create the TestExecute COM object (described here)


    2. Create an in memory XmlDocument from the XML file


    3. foreach <test> element in the XmlDocument


        i. Identify which IntegrationObject operation is required from the <test> type attribute


        ii. Invoke the IntegrationObject operation passing the parameters defined in the child elements, e.g. <projectName>


     


    This isn't particularly difficult but does require a reasonable amount of "plumbing" to make sure it doesn't fall over in a screaming heap.


     


    Regards,


    Phil Baird




  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor
    11 years ago
    I don't believe there is a way to dynamically assign the timeout values on the commandline.



    As for writing the batch file, I think you would simply create a batch file calling out each individually noted Test Item on a line using the commandline structure you've already researched.  I'm not sure, beyond that, what you mean by an "execution list".  



    Hope this helps.
  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor
    11 years ago
    Your "list" is simply a folder containing a bunch of items.   You can't run a "folder".  As mentioned, you'll have to create a batch file which, based upon your screenshot, will have 18 different command lines (or more) that, each command line will call a different test item.



    You then create a different batch file for each "execution list" that you'd need to run.  As mentioned, via commandline, you can't modify the timeout for the test items.  



    Now, I think if you use something like QA Complete or ALM Complete, I think those products have the ability of creating something more like what you're looking for.  I have minimum experience with those products so I'll defer to better experts but, as I understand it, that's what they are intended to do.
  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor
    11 years ago
    See?  I knew someone smarter than me would come along...



    Thanks, Viper!  I've bookmarked those articles for my own use. :-)
  • tristaanogre's avatar
    tristaanogre
    Esteemed Contributor
    11 years ago
    Actually, TestExecute command line should be the same as TestComplete.  AFAIK, there is no difference.  anything you can do calling TestComplete commandline you can do with TestExecute.
  • nnguyen_1's avatar
    nnguyen_1
    Contributor
    11 years ago








    lee hou

    Reputation: 75

    Dear Robert Dear Viper,

    Thanks for your help!! I will try the scripts try out. The basic idea was that i did expect that i could run all my testprojects with testexecute just like testcomplete without any additional effort. It seems like testexecute has only very limited feature..

    Br,

    Li

    I was expecting the same when I came across this issue. Unfortunately, group executing is not available by TE cmd by default yet (or as far as I know). I think it's still acceptable to include additional scripts as my suggestions above to do the job since it's pretty simple. Hopefully the command will be included to TE by default in next versions. Let me know if it works for you. 



    robert Martin: No problem, use as your will ;). I understand Lee Hou's point here when he said  TE is limited.

    - If he creates many groups and enables them in his project, TestComplete has no problem to execute all of those groups by one F5 command. 

    - However, he cannot find the equivalent command to run the same scenerio on TestExecute, which leads him to the question in this topic. 

    With this point, TE has limitations compared with TC. 



  • leeasdf9's avatar
    leeasdf9
    Frequent Contributor
    11 years ago
    Thanks Phil for providing your xml sample, i will try it out with TE via COM. The best thing by Testcomplete automation is that i always get excellent supports and sample scripts. i really appreciate that. thanks again.

    Br,

    Lee