Forum Discussion

groovyguy's avatar
groovyguy
Community Hero
10 years ago

SoapUI NG Pro and a Large Test Suite

Over the last few months I've been working as a software tester for a variety of different webservices. This is new to me, and I've been learning a lot of the In's and Out's of SoapUI. I'm currently using SoapUI NG Pro and am running into some issues. I've been in charge basically of developing the testing plan and the tests themselves to put our code through its paces.

My ultimate plan is to cover as many positive, negative, and NULL test cases as possible: Good data/scenarios, bad data/scenarios, and NULL/no data(some of these tests can be positive, some will be negative, depending on wsdl/schema). For the most part this has worked great for me with the services I've been working on. This latest service has so many optional elements or elements that can be an empty string that to thoroughly test I have 547 NULL test cases that work and 643 that do not work.

 

The question at hand are the test cases that work. There has to be "unique" data to test these conditions to verify that the code can handle as many possible combinations, and I know what I have isn't even a complete set. That being said, I've generated 547 sets of data to test with, I've generated the 547 NULL test cases that will use this data and should work with the code (I love making groovy generate my tests for me), and now SoapUI chokes and hangs.

 

The project appears to simply be too large, too much, for SoapUI NG Pro to handle. My laptop has 32gb of RAM, xmx is set to 16gb, xms is set to 128mb. I am not sure what else I can to work with this level of thorough testing and not cause my laptop to melt when opening this project in SoapUI. Right now I've been waiting for SoapUI to open my project for 5+ minutes. 

 

Has anyone else seen anything like this before? Am I doing something inherently wrong?

9 Replies

  • kondasamy's avatar
    kondasamy
    Regular Contributor
    10 years ago

    Yes ! I have gone through this experience, where my project file would cost around 70 MB of size with my total testcases count to 250 test cases, with each having 200 test steps as an average. I had solved this problem by few of these solutions,

    1) Since my ultimate goal is data driven testing, I separated the data and the testable method and had the dat in external excel sheet and then retained just the necessary method inside SoapUI project file. This gave me an output of just 3 MB project file and with some set of external excel sheets. The time taken to load the SoapUI project has been ideally reduced.

    2) Another method is divide and conquer, without following the data driven approach. I would possibly split my project as 5*50 test cases to result 250 test cases, by grouping them based on common flows. This even reduced and some how stopped this problem.

     

    Finally, I would prefer you to follow the solution 1 which really gave an awesome result.

     

    Thanks,

    Samy

    • groovyguy's avatar
      groovyguy
      Community Hero
      10 years ago

      Okay, what you are saying makes sense to me and I may be able to leverage that for some of my testing. The other part of my testing is also dealing with "missing" data, or when a field is optional. This means that a good portion of my testwill each need their own individual request where the data is not there. Unless there's a way to do this as well?

       

      Some of our NULL tests is to test what happens if the data is blank (empty string), or if the data isn't there. Examples are below. So I think I could potentially use data driven testing for the empty data tests, but have seperate requests to handle each possible element being missing. Thoughts? 

       

       

      Empty Data:

      <Element></Element>

      or

      <Element/>

       

      Missing Data

      <!-- <Element/> --> 

      • kondasamy's avatar
        kondasamy
        Regular Contributor
        10 years ago

        Possibly yes! It all lies in the way we are framing the request XML through the data drive approach. I just share my experience in handle this scenario through the below ways,

        1) If I leave my cell (Excel cell which holds data) is empty/ null, then probably my customized code should either comment/ pass nothing; which again sends as empty tags (<Element></Element>) or commented tag (<!-- <Element/> --> ), which in no way recognzed by the service.

        2) Another method is to feed some specific keywords as ***EMPTY*** in the excel cells. So, that my customized code will consider it as a sign for passing empty tag and would do so.

        3) We can even pass the request XML taking directly from an excel sheet/ flat files.

         

        I know there are better ways than this. I hope others would share there points too!

         

        Thanks,

        Samy

    • nmrao's avatar
      nmrao
      Icon for Champion Level 1 rankChampion Level 1
      10 years ago
      The approaches are good. 200 steps(over-exaggerated) in a case?
      • kondasamy's avatar
        kondasamy
        Regular Contributor
        10 years ago

        @nmrao: The approaches are good. 200 steps(over-exaggerated) in a case?

         

        Yes, it is is little large! But, it is very small when I compare to the test case which even test for 200000 test data. A data intensive test case though.:smileyhappy:

         

        Thanks,

        Samy