Running a subset of tests in a project

[jose_pjoseph]

I have a project with 100 tests which I have set up with Jenkins to run on a schedule. I would like to run 2 jobs daily:

1. A "smoke" job which runs 30 tests out of the 100 tests twice a day.
2. A second job which runs all the 100 tests once a day.

How can I group the "smoke" tests in a project as mentioned above? I found "How to Run a Group of Tests", but that talks about ordering the tests and nothing about running a specific group of tests.

[Marsha_R]

Have one TC project for just the 30 smoke tests and a separate one for the others.  Keep both those projects in the same project suite.  Then either project can be run separately or the whole project suite can be run.

[jose_pjoseph]

Hmm. The one project I gave above was just an example. We have multiple projects already in our test suite. So this will involve some rework and process changes moving forward. Also, I don't think it will be a good place to be in future when we have more tests and need to re-organize tests from either group, like moving tests from one group to the other.

Is this the only way? Is there no way to tag a test in a project to belong to a group and then execute the group?

[Marsha_R]

Rather than an example, it's usually better if you can give us specifics.  Tell us exactly what you have and exactly what your goal is.  Otherwise we go around in these "that's not quite what I wanted" circles and that doesn't get you an answer.

[jose_pjoseph]

I want to run a subset of tests in a project (without creating a new project for them, if possible).

More specifically, I have a project suite with 5 projects:

1. Project 01 - 70 tests
2. Project 02 - 120 tests
3. Project 03 - 68 tests
4. Project 04 - 50 tests
5. Project 05 - 85 tests

Running the project suite and all the 393 tests in it takes a long time and doing it more than once a day is not very effective. Sot I want to run a subset of the higher priority tests more frequently, which will give me a good idea about the state of my web app through the day and will also catch any major issues. These are the subset of tests that have been identified for that:

1. Project 01 - 25 tests (out of 70 tests total)
2. Project 02 - 40 tests (out of 120 tests total)
3. Project 03 - 20 tests (out of 68 tests total)
4. Project 04 - 15 tests (out of 50 tests total)
5. Project 05 - 20 tests (out of 85 tests total)

That adds up to 120 out of the 393 tests. As I mentioned in my initial description, I want to create 2 Jenkins jobs, one of them running 393 tests once a day and the other running 120 tests twice (or more) a day. I am looking for a way to execute the 120 tests in the above scenario. Hope the requirement is clear. Do let me know if there is still some confusion.

[Marsha_R]

@tristaanogre, @cunderw Any thoughts?

[tristaanogre · Accepted solution]

I'm not very familiar with the integration between TestComplete and Jenkins... however, as a command line, you can tell TestComplete/TestExecute to run a test item.  So, if you create a "parent" test item that "contains" the ones that you want to run, you can call that via command line.  See https://support.smartbear.com/testcomplete/docs/working-with/automating/command-line-and-exit-codes/... and reference the "/test:" command line switch.

If you organize your tests appropriately, you can call a batch that way. 

[cunderw · Accepted solution]

If test items are being used for your project under organize tests then you can group your test items under a parent.

So test items for Project 01 would look like:

• SmokeTest
  • Test 1
  • Test 2
  • Test ...
  • Test 25
• Test 26
• Test ...
• Test 70

Then on jenkins for your smoke tests job you can specify Project Test when setting up the Test Complete step and use SmokeTest.

For the full run job you can then do the entire suite option

[jose_pjoseph]

Thanks @tristaanogre & @cunderw! /test:<TestItem> does what I need. I will try that in my Jenkins setup and see if I can make the Jobs work.