Forum Discussion

khanchett's avatar
khanchett
Contributor
6 years ago
Solved

When doing an assert on a large response (10,000-50,000 lines) SoapUI is taking a long time 5+mins.

I have a response that is anywhere between 10,000 and 50,000 lines.  I have a few asserts that check the uniqueness of those values.  I understand this will take some time, but is there anyway to designate more resources to SoapUI so that these don't take forever?  

 

I've had a 46,000 line that takes almost 15min to process one assert!

 

I am on the latest release of SoapUI. I've also played with the testrunner.bat and ready-api.bat files and un-remmed the line and upped the value for:

 

set READY_XMX=16000m

along with 

Xms512m

 

Appreciate any help in this matter!

  • The groovy script I provided can be easily adapted to check against multiple expected form names. To address your latest concern, I have noticed the same behavior. Whenever you edit/change a groovy script assertion and close the window, it reprocesses that assertion and re-runs it against the response. There's no real way around that unfortunately. I am not sure why it would take so much longer. 

9 Replies

  • groovyguy's avatar
    groovyguy
    Champion Level 1

    What sort of assertions are you using to process the test and verify it passes? 

  • I perform an xPath Match assertion to count the values of a tag matching a specific value based on project variables.  Normally this isn't a problem, but when the line sizes get beyond 10000 I've had it lock up SoapUI and the only resolution is to kill the process and start over.  I've let it run for 20-30 min.before doing so.

    • groovyguy's avatar
      groovyguy
      Champion Level 1

      Depending on how you have written your xpath statement, this could cause it to try to scan the entire response inefficiently which can lead to the issue you are seeing where ReadyAPI locks up.

       

      When I have a response that large to parse, I tend to shift to using groovy to verify the response versus the request made. I can have better control, especially if the response returns more than 1 object. 

  • The issue is in order to confirm the values it needs to read all the XML.  I wouldn't think this is an unusual request. 

     

    I'm not a novice but I'm not an expert either.  Below is my assertion, can you see a better way to do this assert?

     

    declare namespace ns1='http://xxx.org/';
    declare namespace a='http://xxxxx.V3.ServiceTypes';
    (count(//a:Coverages[1]/a:Coverage/a:Form/a:Name[text() = "${#TestSuite#FormName13}"]) = count(//a:Coverages[1]/a:Coverage/a:Form/a:Name))
    and
    (count(//a:Coverages[1]/a:Coverage/a:Form/a:Name)>0)

     

    The first part:  Counts the possible form names, every coverage MUST have a form name.  So if there is only one kind of form name then every coverage should be accounted for.  Sometimes there is more than one and I take that into account.

    The second part:  confirms there are coverages, otherwise it should fail.

     

    Would it be better to split these up into separate assertions? I'm not convinced this wouldn't be a problem still.

     

    Thanks for any advice!

    • groovyguy's avatar
      groovyguy
      Champion Level 1

      So if I understand your assertion correctly, and if I understand xpath in general, here's what you are saying:

       

      (Find the count of every Name object whose text is equal to ${#TestSuite#FormName13}) and compare that to (Find the Count of how many Coverages[1]/Coverage/Form/Name objects there are)

      Here, you are finding the count of all the objects twice. I am going to make some assumptions here to generalize the problem. For 10000 lines, assuming 1 object per line, that's parsing 20000 lines twice. Assume each 10 lines takes 1 second. That's 10000 lines for 100 seconds, twice, so 200 seconds. Again, just generic numbers and assumptions, but it should show the case that you are counting all 10000 lines/objects twice.

      An easier way may be to write a groovy script assertion. When I typically need to tackle a problem like this, I do so with groovy. Groovy scripts can be very powerful, but require some knowledge of scripting/programming languages. I could assist if that is something you want to work with. 

  • So here is something interesting.  When the test runs those assertions don't take as long as if I edit the assertion after a run. 

     

    I updated the previous assertion to groovy and it still takes a really long time after an edit.

     

    def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
    def responseHolder = groovyUtils.getXmlHolder(messageExchange.responseContent)
    Boolean flag = true;
    
    // Count all form name tags
    int formNamesCount = Integer.parseInt(responseHolder.getNodeValue("count(//a:Coverages[1]/a:Coverage/a:Form/a:Name)"))
    
    //Count all the forms with a match to give names from relevant test suite property 
    def formName = context.expand( '${#TestSuite#FormName3}' )
    int formNamesCount2 = Integer.parseInt(responseHolder.getNodeValue("count(//a:Coverages[1]/a:Coverage/a:Form/a:Name[text() ='" + formName + "'])"))
    if((formNamesCount != formNamesCount2) ||  (formNamesCount<=0)){
    	flag = false
    }
    assert flag

    So the steps to recreate this is the following:

    1. I run my test step, this will execute all the assertions that I have and it doesn't take super long.
    2. I find the one assert to change based on something I see in the response to update.
    3. I open the XPath Match or, now updated, Script Assertion.
    4. I edit whatever and click the test button, it responds quickly.
    5. Now when I do one or the other I get the long delay.
      1. XPath Match - clicking SAVE it will hang the screen until it's finished.
      2. Script Assertion - clicking OK it will return to the Assertions listing but SoapUI is completly locked until it's finished.

    So the main question is what is different from the run or clicking the test button, to the actual saving of the assertion, whichever it is.

     

    Hopefully this explains the issue best.  I was hoping the Groovy script was the answer. 

     

    If you see anything in the code I can change to improve it I'm all ears.  :smileywink:  I appreciate all the feedback!

     

    Thanks!

     

    • groovyguy's avatar
      groovyguy
      Champion Level 1

      I am going to build off of your groovy script a little bit and see if I can help. Looking at your script, you are still counting everything nultiple times, which still duplicates your effort. 

       

      def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
      def responseHolder = groovyUtils.getXmlHolder(messageExchange.responseContentAsXml)
      Boolean flag = true;
      
      // Count all form name tags
      int formNamesCount = responseHolder["count(//a:Coverages[1]/a:Coverage/a:Form/a:Name)"].toInteger();
      
      // Get all form name tags
      def formNames = responseHolder["//a:Coverages[1]/a:Coverage/a:Form/a:Name"]
      
      def formName = context.expand( '${#TestSuite#FormName3}' )
      
      // Test if there is only one object in response
      if (formNamesCount == 1)
      {
      	// Only one formName returned
      	assert (formNames.equals(formName)
      }
      else if (formNamesCount > 1)
      {
      	// More than one formName returned
      	assert (formNames.count{ it.equals(formName) } == formNamesCount)
      }
      else 
      {
      	// Zero items returned condition
      	// Handle this as you wish, I usually assume a failure as I expect data returned
      	assert false;
      }
       
      
  • So the form name can be 1 or more form names that I'm comparing.  So lets change the scenario and say I have 3 different form name property/variables: name1, name2 and name3.  Also lets say my form's count (counting up all the xpath form/names) is 50 (as an example). 

    I'm totaling up count(name1) + count(name2) + count(name3) to make sure all the form/name combinations are accounted for.  So checking if it's equal to the overall xpath form/names count confirms this.  I couldn't think of a better way to make sure all my tags have a value and a correct value.

     

    I was trying to keep my example simple, but the assertion can get a little more complex.  I'm wondering if the better fix would be to split the 2 checks into 2 separate assertions?  Put the if count>0 in one assertion and then verify all the other sums of form name counts in another assertion.  I just feel the latter will still have somewhat of an issue.

     

    It's just odd the running of the test case or the testing inside the assertion runs quick.  But when I save out of editing it takes so long.

     

    Thanks!

    • groovyguy's avatar
      groovyguy
      Champion Level 1

      The groovy script I provided can be easily adapted to check against multiple expected form names. To address your latest concern, I have noticed the same behavior. Whenever you edit/change a groovy script assertion and close the window, it reprocesses that assertion and re-runs it against the response. There's no real way around that unfortunately. I am not sure why it would take so much longer.