cancel
Showing results for 
Search instead for 
Did you mean: 

Running a subset of tests in a project

SOLVED
Highlighted
Contributor

Running a subset of tests in a project

I have a project with 100 tests which I have set up with Jenkins to run on a schedule. I would like to run 2 jobs daily:

  1. A "smoke" job which runs 30 tests out of the 100 tests twice a day.
  2. A second job which runs all the 100 tests once a day.

How can I group the "smoke" tests in a project as mentioned above? I found "How to Run a Group of Tests", but that talks about ordering the tests and nothing about running a specific group of tests.

2 ACCEPTED SOLUTIONS

Accepted Solutions
Highlighted
Community Hero

Re: Running a subset of tests in a project

I'm not very familiar with the integration between TestComplete and Jenkins... however, as a command line, you can tell TestComplete/TestExecute to run a test item.  So, if you create a "parent" test item that "contains" the ones that you want to run, you can call that via command line.  See https://support.smartbear.com/testcomplete/docs/working-with/automating/command-line-and-exit-codes/... and reference the "/test:" command line switch.

 

If you organize your tests appropriately, you can call a batch that way. 


Robert Martin
[Hall of Fame]
Please consider giving a Kudo if I write good stuff
----

Why automate?  I do automated testing because there's only so much a human being can do and remain healthy.  Sleep is a requirement.  So, while people sleep, automation that I create does what I've described above in order to make sure that nothing gets past the final defense of the testing group.
I love good food, good books, good friends, and good fun.

Mysterious Gremlin Master
Vegas Thrill Rider
Extensions available

View solution in original post

Highlighted
Community Hero

Re: Running a subset of tests in a project

If test items are being used for your project under organize tests then you can group your test items under a parent.

 

So test items for Project 01 would look like:

  • SmokeTest
    • Test 1
    • Test 2
    • Test ...
    • Test 25
  • Test 26
  • Test ...
  • Test 70

 

Then on jenkins for your smoke tests job you can specify Project Test when setting up the Test Complete step and use SmokeTest.

 

For the full run job you can then do the entire suite option


Thanks,
Carson

Click the Accept as Solution button if my answer has helped

View solution in original post

8 REPLIES 8
Highlighted
Community Hero

Re: Running a subset of tests in a project

Have one TC project for just the 30 smoke tests and a separate one for the others.  Keep both those projects in the same project suite.  Then either project can be run separately or the whole project suite can be run.

Highlighted
Contributor

Re: Running a subset of tests in a project

Hmm. The one project I gave above was just an example. We have multiple projects already in our test suite. So this will involve some rework and process changes moving forward. Also, I don't think it will be a good place to be in future when we have more tests and need to re-organize tests from either group, like moving tests from one group to the other.

Is this the only way? Is there no way to tag a test in a project to belong to a group and then execute the group?

Highlighted
Community Hero

Re: Running a subset of tests in a project

Rather than an example, it's usually better if you can give us specifics.  Tell us exactly what you have and exactly what your goal is.  Otherwise we go around in these "that's not quite what I wanted" circles and that doesn't get you an answer.

Highlighted
Contributor

Re: Running a subset of tests in a project

I want to run a subset of tests in a project (without creating a new project for them, if possible).

 

More specifically, I have a project suite with 5 projects:

  1. Project 01 - 70 tests
  2. Project 02 - 120 tests
  3. Project 03 - 68 tests
  4. Project 04 - 50 tests
  5. Project 05 - 85 tests

Running the project suite and all the 393 tests in it takes a long time and doing it more than once a day is not very effective. Sot I want to run a subset of the higher priority tests more frequently, which will give me a good idea about the state of my web app through the day and will also catch any major issues. These are the subset of tests that have been identified for that:

 

  1. Project 01 - 25 tests (out of 70 tests total)
  2. Project 02 - 40 tests (out of 120 tests total)
  3. Project 03 - 20 tests (out of 68 tests total)
  4. Project 04 - 15 tests (out of 50 tests total)
  5. Project 05 - 20 tests (out of 85 tests total)

That adds up to 120 out of the 393 tests. As I mentioned in my initial description, I want to create 2 Jenkins jobs, one of them running 393 tests once a day and the other running 120 tests twice (or more) a day. I am looking for a way to execute the 120 tests in the above scenario. Hope the requirement is clear. Do let me know if there is still some confusion.

Highlighted
Community Hero

Re: Running a subset of tests in a project

@tristaanogre@cunderw Any thoughts?

Highlighted
Community Hero

Re: Running a subset of tests in a project

I'm not very familiar with the integration between TestComplete and Jenkins... however, as a command line, you can tell TestComplete/TestExecute to run a test item.  So, if you create a "parent" test item that "contains" the ones that you want to run, you can call that via command line.  See https://support.smartbear.com/testcomplete/docs/working-with/automating/command-line-and-exit-codes/... and reference the "/test:" command line switch.

 

If you organize your tests appropriately, you can call a batch that way. 


Robert Martin
[Hall of Fame]
Please consider giving a Kudo if I write good stuff
----

Why automate?  I do automated testing because there's only so much a human being can do and remain healthy.  Sleep is a requirement.  So, while people sleep, automation that I create does what I've described above in order to make sure that nothing gets past the final defense of the testing group.
I love good food, good books, good friends, and good fun.

Mysterious Gremlin Master
Vegas Thrill Rider
Extensions available

View solution in original post

Highlighted
Community Hero

Re: Running a subset of tests in a project

If test items are being used for your project under organize tests then you can group your test items under a parent.

 

So test items for Project 01 would look like:

  • SmokeTest
    • Test 1
    • Test 2
    • Test ...
    • Test 25
  • Test 26
  • Test ...
  • Test 70

 

Then on jenkins for your smoke tests job you can specify Project Test when setting up the Test Complete step and use SmokeTest.

 

For the full run job you can then do the entire suite option


Thanks,
Carson

Click the Accept as Solution button if my answer has helped

View solution in original post

Highlighted
Contributor

Re: Running a subset of tests in a project

Thanks @tristaanogre & @cunderw! /test:<TestItem> does what I need. I will try that in my Jenkins setup and see if I can make the Jobs work.

New Here?
Join us and watch the welcome video:
Announcements
Join TechCorner Challenge!
Want a fun and easy way to learn TestComplete? Try solving weekly TechCorner challenges and get into the Leaderboard!

Challenge Status

Changing options from outside of TestComplete

Participate!

How to execute remote test and obtain results via Test Runner REST API

Participate!

Comparing content of HTML table with Excel file data

Participate!

Compare HTML table with Excel file and correct data in the file

Participate!
Top Kudoed Authors