Forum Discussion

lowres's avatar
lowres
Occasional Contributor
27 days ago
Solved

Grown up way of handling keyword tests/projects and all their variables.

Hello,

We're running automated tests on a financial application, where input values are processed through business logic to produce an expected output. As the app evolves, input/output values change frequently based on the time of month, entities involved, and updates to business rules.

Currently, we manage test variables using Excel—each Keyword Test has its own sheet containing:

  • Metadata (creator, module, etc.)
  • Test-specific parameters (data views, scenario selection, forms)
  • Input values and expected output

This setup works for our ~10 tests, but I'm exploring more scalable and collaborative solutions. Is Excel still viable for teams with growing test suites, or is a test management tool or web app of some sort better suited?

(Note we are not software developers.)

Thanks for any suggestions.

  • scot1967's avatar
    scot1967
    27 days ago

    Hi lowres,

    There are a few options, AI Test Data Generation may allow you to reduce your dependency on spreadsheets.  I am not sure this would work if you need known custom data. The use of a database would be another good option if you need to store larger sets of custom data.    Managing a lot of spreadsheets sounds like it would quickly become a burden if you go out to several hundred tests.  You could contain your test data in several sheets within the same Excel file to relieve some of the file management burden.  However you may hit data access issues if multiple testers are working from the same data file.  The separate Excel file method will hit this wall even with separate files.   Databases don't have this issue. 


    If you learned SQL and a scripting language you could supplement your KWTs and they would scale much better.  This would allow those less skilled to continue using KWT and call the functions that are created in script.  There are many nice options available.  JavaScript and Python are very popular and the learning curve isn't too bad.  SQL isn't too bad for the simple things you will likely need.  There are free SQL options out there and learning to script costs you nothing but your time.

    Microsoft® SQL Server® 2022 Express:


    There are limits here but you can expand to a paid version as you needs expand.


    ... If you find my posts helpful drop me a like! 👍 Be sure to mark or post the solution to help others out and/or to credit the one who helped you. 😎


6 Replies

  • scot1967's avatar
    scot1967
    Icon for Champion Level 2 rankChampion Level 2
    27 days ago

    Hi lowres,

    There are a few options, AI Test Data Generation may allow you to reduce your dependency on spreadsheets.  I am not sure this would work if you need known custom data. The use of a database would be another good option if you need to store larger sets of custom data.    Managing a lot of spreadsheets sounds like it would quickly become a burden if you go out to several hundred tests.  You could contain your test data in several sheets within the same Excel file to relieve some of the file management burden.  However you may hit data access issues if multiple testers are working from the same data file.  The separate Excel file method will hit this wall even with separate files.   Databases don't have this issue. 


    If you learned SQL and a scripting language you could supplement your KWTs and they would scale much better.  This would allow those less skilled to continue using KWT and call the functions that are created in script.  There are many nice options available.  JavaScript and Python are very popular and the learning curve isn't too bad.  SQL isn't too bad for the simple things you will likely need.  There are free SQL options out there and learning to script costs you nothing but your time.

    Microsoft® SQL Server® 2022 Express:


    There are limits here but you can expand to a paid version as you needs expand.


    ... If you find my posts helpful drop me a like! 👍 Be sure to mark or post the solution to help others out and/or to credit the one who helped you. 😎


    • lowres's avatar
      lowres
      Occasional Contributor
      27 days ago

      Thanks Scot, OK so no out of the box solution, Im going to have to do some work, phfff :D.

      I have been looking this morning after talking to support at using a db (MSSQL), and putting a web UI on top of it. 

      It suddenly become more complicated since we now need to use a Table Checkpoint to check a list of  Forms are present. Easy with Excel, needs some design consideration to include this in the Database design.

      Its going to take some testing to see what works and what is manageable. 

    • lowres's avatar
      lowres
      Occasional Contributor
      27 days ago

      Thanks Rraghvani, Im looking at DB now and continuing using Excel int he short term, see if it really is an issue or I am just overly concerned.

      • rraghvani's avatar
        rraghvani
        Icon for Champion Level 3 rankChampion Level 3
        27 days ago

        Our manual testing process currently relies on Excel spreadsheets, which serve as our primary test documentation. These spreadsheets include the following headers: Acceptance Reference Number, Test Description, Expected Outcome, Test Number, Test Date, Tester Name, and Environment/Build Number.

        While using Excel may seem outdated, this approach is intentional. Our testing often involves complex actuarial calculations related to insurance, pensions, and finance, and Excel allows us to easily manage and reference extensive test data alongside those calculations.

        The system under test is a web-based application with a backend database. For automation, we provide specific inputs and verify that the outputs match the expected results. Because the same set of test data is used consistently, the expected outputs should remain unchanged across test runs.

  • Hassan_Ballan's avatar
    Hassan_Ballan
    Icon for Champion Level 3 rankChampion Level 3
    27 days ago

    It all depends on the source of your data and you have many options https://support.smartbear.com/testcomplete/docs/testing-with/data-driven/organizing-storages.html

    As you mention you are not software developers, I would also assume that coding or database access would also be an additional challenge. My preferred method is to stick with Excel. Things you can do to improve handling KeywordTests and variables to scale.

    1. Consider consolidating data to centralize, this makes data maintenance easier and reduces duplication.
    2. Leverage TestComplete's Data-Driven Testing.
    3. Parameterize your Keyword Tests.
    4. Define & Use Project and Project Suite Variables Wisely. Use Project Variables for test-level data. Use Project Suite Variables for data shared across tests.
    5. Using Excel Files as Data Storage https://support.smartbear.com/testcomplete/docs/testing-with/data-driven/excel-storages.html  DB Table variables https://support.smartbear.com/testcomplete/docs/testing-with/variables/data-types/db-table.html 

    💬 Found the answer helpful? Give it a Kudos by clicking Like!
    ✅ Got your issue resolved? Click Mark as Solution so others can find it quickly.