Forum Discussion

sparkins's avatar
sparkins
Occasional Contributor
9 years ago

What is the recommended size of a test set?

We have a test set that has >1000 tests and it performs very poorly particularly for remote users.  I am look at the best way to divide it up and I'm wondering what the recommend size (or ceiling) is?

  • DWalsh's avatar
    DWalsh
    Moderator
    9 years ago

    Hi

     

    1000 test sets is quite allot. Can you tell me the requirements for having such a large number of test sets? Are you able to break these down further say by builds, releases, functions, features etc. Having smaller number like anything will reduce the overhead

     

    Kind Regards

    Damien

     

    Technical Sales Engineer

    • sparkins's avatar
      sparkins
      Occasional Contributor
      9 years ago

      HI Damien,

       

      I meant we have one regression test set that has >1000 tests in it that we manually test with each release.  The reason we have it set up that way (legacy from the test cases tab, but have just migrated to Test Management>Test Library) is we are easily able to modify the test assignments within that one test set, as we often have a different person test it each release.  We may break it out by tester, but then we have to move tests around within each test set each release, which is not ideal.  If we break it out by Feature, then there is no easy way for each tester to get a test run of all the tests that they are responsible for.  We may be able to circumvent that by using reports and dashboards.  I know there are some updates coming to the test runner, so we'll see if that helps at all.

       

      Do you have a number of tests in a test set thats recommended, so as a remote tester I wont have performance issues?

       

      Thanks

      Simon