Thread Safety issues in Mock Service as WAR
I am working with a client and was hoping to propose the use of SoapUI v3.6.1 mock services as a viable alternative to hand coded mock services. They use these mock services to provide stubs for back end functionality and to eliminate the back end when doing performance testing and analysis of the middle tier.
Given the problems we're seeing I would have a hard time, at this time, recommending the use of SoapUI to my client.
Before making this recommendation to the client, we wanted to perform some due diligence to make sure that SoapUI mock services could scale and not be a bottleneck. Our test methodology was to create a simple flat load model using LoadUI and direct it at the SoapUI generated Mock deployed as a WAR in a JavaEE container. We are not trying to use any of the "logging" features of the SoapUI mock, just simply wanted to see that it could respond in a timely manner under load.
What we found was that it started to throw NullPointerException under relatively low load. (A "scrubbed" stack trace is at the bottom of this email after my analysis).
Looking at the source code the fundamental issue has to do with the fact that the generated WAR file uses instance level variables to track "results" as well as a raft of other things. No locking or synchronization is done to make this thread safe, so after a fairly trivial amount of load, the servlet begins to fail, as it's trying to do multiple modifications to the list elements from different threads. The solution is to either make this thread safe through synchronization / locking (which of course impacts performance) or modify the manner in which you're tracking historical data.
There has been at least one other report on these forums of problems which is related to this specific issue. Here's link to that report:
http://www.eviware.com/forum/viewtopic.php?f=13&t=4651&p=15440&hilit=war#p15440
Here's the scrubbed stack trace from our issue:
Your attention to this issue would be greatly appreciated!
Thank you,
Macon Pegram
Given the problems we're seeing I would have a hard time, at this time, recommending the use of SoapUI to my client.
Before making this recommendation to the client, we wanted to perform some due diligence to make sure that SoapUI mock services could scale and not be a bottleneck. Our test methodology was to create a simple flat load model using LoadUI and direct it at the SoapUI generated Mock deployed as a WAR in a JavaEE container. We are not trying to use any of the "logging" features of the SoapUI mock, just simply wanted to see that it could respond in a timely manner under load.
What we found was that it started to throw NullPointerException under relatively low load. (A "scrubbed" stack trace is at the bottom of this email after my analysis).
Looking at the source code the fundamental issue has to do with the fact that the generated WAR file uses instance level variables to track "results" as well as a raft of other things. No locking or synchronization is done to make this thread safe, so after a fairly trivial amount of load, the servlet begins to fail, as it's trying to do multiple modifications to the list elements from different threads. The solution is to either make this thread safe through synchronization / locking (which of course impacts performance) or modify the manner in which you're tracking historical data.
There has been at least one other report on these forums of problems which is related to this specific issue. Here's link to that report:
http://www.eviware.com/forum/viewtopic.php?f=13&t=4651&p=15440&hilit=war#p15440
Here's the scrubbed stack trace from our issue:
####<Jan 5, 2011 3:23:23 PM EST> <Error> <HTTP> <XXXXXXX-XXXXXXX> <XXXXXXX> <[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1294259003274> <BEA-101020> <[weblogic.servlet.internal.WebAppServletContext@411af96 - appName: 'XXXXXMockServices', name: 'XXXXXMockServices', context-path: '/XXXXXMockServices', spec-version: '2.5'] Servlet failed with Exception
java.lang.NullPointerException
at org.apache.commons.collections.list.TreeList.remove(TreeList.java:222)
at com.eviware.soapui.mockaswar.MockAsWarServlet$MockServletSoapUICore.dispatchRequest(MockAsWarServlet.java:233)
at com.eviware.soapui.mockaswar.MockAsWarServlet.service(MockAsWarServlet.java:171)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3498)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2086)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1406)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
>
Your attention to this issue would be greatly appreciated!
Thank you,
Macon Pegram
