Forum Discussion

SKoneru's avatar
SKoneru
Contributor
12 years ago

Tagging tests

Hi Support team,

I am able to create a custom junit report and run all the tests related to a specific project.
I relied on "maven-soapui-extension-plugin" and did provide the required inputs like configuring the custom report in the configuration section for "soapui.junit.reportCollector" property.

Now, there is a new request to tag the tests. This is exactly like the groups concept in testng framework.
I want to run the tests that are tagged for a specific domain and/or a service.
Once I specify the domain and/or service, then SOAPUI-PRO should be able to fetch and run all the tests(which might be in different projects in different test suites) that are tagged with that specific domain and service.

How can I hook up testng and rely on SoapUIProExtensionTestCaseRunner to make all the tests in different projects to get kicked-off and generate the custom junit report(as I had already created one).

Thanks
Surya
  • redfish4ktc2's avatar
    redfish4ktc2
    Super Contributor
    12 years ago
    Could you confirm what I have understood about your question ?

    1st, I think your request is not directly related the maven plugin, nor to the junit report collector
    You want to be able to run
    * for a given project, only some tests (testcases?) based on some criteria (so you want to know how to define the criteria in the tests and how to tell soapui to use these criteria)
    * same than the previous but in various project
  • SKoneru's avatar
    SKoneru
    Contributor
    12 years ago
    >>(so you want to know how to define the criteria in the tests and how to tell soapui to use these criteria)
    To define the criteria, I was relying on tagging(groups) concept in testng.
    I should be able to clarify each testcase in all the testsuites and in all the projects with some standard notation. For example, let's say we have two projects ProjectA, ProjectB.
    ProjectA has Testsuites TS1 and TS2 and testcases in TS1 are TC1, TC11 and testcases in TS2 are TC2, TC21.
    ProjectB has Testsuites TS3 and TS4 and testcases in TS3 are TC3, TC31 and testcases in TS4 are TC4, TC41.

    Now I will define a criteria or groups
    DOMAIN={SELLING,BUYING}, PRIORITY={P1,P2}, SERVICE={BUYINGSERVICE,TRADINGSERVICE}
    I should can tag each test case with one or more of these groups.

    Let's say,
    TC1 is tagged as DOMAIN={BUYING}, PRIORITY={P1}
    TC11 is tagged as DOMAIN={SELLING}, PRIORITY={P1}, SERVICE={BUYINGSERVICE}
    TC2 is tagged as DOMAIN={SELLING}, PRIORITY={P1},SERVICE={BUYINGSERVICE}
    TC21 is tagged as DOMAIN={BUYING}, PRIORITY={P2}
    TC3 is tagged as DOMAIN={SELLING}, PRIORITY={P1}, SERVICE={BUYINGSERVICE}
    TC31 is tagged as SERVICE= {TRADINGSERVICE}
    TC4 is tagged as DOMAIN={BUYING}, PRIORITY={P2}
    TC41 is tagged as DOMAIN = {SELLING}, PRIORITY={P2}, SERVICE={TRADINGSERVICE}

    Now when I want to execute tests via maven plugin or TestNg route, I should be able to say
    Run all the tests in PROJECTA, PROJECTB that are tagged as DOMAIN={SELLING} and SERVICE={BUYINGSERVICE} - This should run TC11, TC2, TC3
    Run all the tests in PROJECTA, PROJECTB that are tagged as SERVICE={TRADINGSERVICE} - This should run TC31 and TC41
    Run all the tests in PROJECTA, PROJECTB that are tagged as PRIORITY={P2} - This should run TC4, TC41, TC21

    We are not confined to one Project, we should be able to run tests that are in different projects. The test cases in those projects are picked based on the tags we opted.
    We can have this as plugin extension or want to see if we can achieve this via testng.

    This is very important for us. Please let me know if we can achieve this..
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    12 years ago
    Hi,

    Thanks for bringing this up. Unfortunately currently SoapUI does not support tagging. But it is a really an interesting suggestion and I will take this to our product owner for consideration.

    In the meantime, you have to manually pass in the test suites/cases name to the maven plugin to achieve what you are planning to do through tagging.

    Regards,
    Shadid
    SmartBear Sweden.
  • SKoneru's avatar
    SKoneru
    Contributor
    12 years ago
    >>In the meantime, you have to manually pass in the test suites/cases name to the maven plugin to achieve what you are planning to do through tagging.
    I am currently relying on "maven-soapui-extension-plugin".
    Is there a way to pass multiple projects for execution and aggregate the test results? Please let me know how this can be done.

    As I couldn't find any solution for this, I ended up posting on maven-soapui-extenstion-plugin forum :- https://github.com/redfish4ktc/maven-so ... /issues/41
  • redfish4ktc2's avatar
    redfish4ktc2
    Super Contributor
    12 years ago
    Hi, maven-soapui-extension-plugin is not supported by SmartBear but by me and only in my free time (I don't have a lot these days)
    Currently, the soapui testcase runner (in any plugin ) only supports a single project

    For your tagging question, I have the following suggestion: you could use test cases properties and a groovy script.
    in each test case, use the properties for tagging
    in a master groovy script (runned before test suites of your project), find a way to retrieve the tags you want to use (java system properties, global properties, ...), loop over all test cases and desactivate those which don't match the tags

    You would have to do this in all project
  • SKoneru's avatar
    SKoneru
    Contributor
    12 years ago
    Hi, Thank you for the suggestion and also for providing the extension plugin.

    The logic that you explained for tagging is good. Can we invoke the groovy script that you suggested to execute as part of ProjectRunListener.beforeTestSuite event. In which case for each project I can hook up this event, instead of invoking this script as part of SetUp Script for each test suite?

    But the power of tagging is best achieved when we can execute test cases from multiple projects that match the tag criteria. So I would still like to have this feature.
  • redfish4ktc2's avatar
    redfish4ktc2
    Super Contributor
    12 years ago
    SKoneru I cannot answer to your ProjectRunListener question. I'm not really familiar with this. Sorry
    SmartBear Support?
  • SKoneru's avatar
    SKoneru
    Contributor
    12 years ago
    Hi Support team,

    Will the logic proposed by me work for the suggestion given by "redfish4ktc2"?

    Thanks
    Surya
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    12 years ago
    Hi Surya,

    It is possible to enable/disable test cases/test steps through the ProjectRunListener.beforeTestSuite event, so that should work just fine We have noted the feature request regarding tagging, and it is added to the backlog for prioritization.

    Regards,
    Arian
    SmartBear Sweden