Forum Discussion

CRicketts's avatar
CRicketts
Occasional Contributor
13 years ago

TPS with Rest protocol is not as expected but works withSOAP

Hi,

I am trying to run tests using SOAPUI Pro on our web service which is configured to work with Rest and SOAP protocols. Issue is after running the tests with Rest protocol the transactions per second obtained is very less but when tested on SOAP service it is on the higher side. The Rest web service was tested using another tester and the throughput obtained is in line with SOAP service. Can you please confirm if there is a difference in how the tps is calculated for REST and SOAP. As on looking at the server logs it was noticed that Rest requests were delayed. Even if the avg response time is same for Rest and SOAP requests the tps obtained for SOAP is 100 times higher than Rest.

The test parameters used were as follows
- 1 thread used
- 10 Total Runs
- Test Strategy used:Simple
- No delay between messages
- Operation used was retrieving a resource

Kind Regards
Amruta

7 Replies

  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    13 years ago
    Hi Amruta,

    I have run a test with a SOAP and a REST interface on the same server, to retrieve the same resource (just a few Kb).
    My results were pretty much as expected:

    SOAP


    REST


    I must assume that there is something unusual about your setup which accounts for the large difference you're having, as REST and SOAP basically should have very similar performance (but of course, that can vary according to your server settings, caching, compression, encryption etc).

    Please do some tests for that, for example by varying the size of the payload, and see how that affects your tests.

    Good luck,

    Renato Athaydes
    SmartBear Software
  • CRicketts's avatar
    CRicketts
    Occasional Contributor
    13 years ago
    Thanks for your replies.

    Even on further testing the results remained same for me. But one thing I would like to mention is that when multiple threads are used the results look good. But on using single thread, which means each request will start after the first is executed the results just drop for tps calculation. On the server side it was noticed that requests were received after a longer delay. I think that is what is affecting the tps calculation. Will post some results soon.

    Your help is very much appreciated.

    Kind Regards
    Amruta
  • CRicketts's avatar
    CRicketts
    Occasional Contributor
    12 years ago
    Hi,

    I performed more test and the result was no different. Looking at the response times(given below in milliseconds) they are very close to each other which suggest that the request itself is taking taking more time for Rest protocol than on a SOAP request.

    As mentioned earlier. The tps are good when the the tests are carried out using multiple threads. Also test using other tester showed similar results for Rest and Soap. As we are using SoapUI as a preferred testing tool we are keen on getting this issue sorted.

    Would it help if I provide a url for test and then you can do some tests from your side?

    Also is there a setting to log times along with ms as that would be helpful to do some more comparison

    Many Thanks
    Amruta
  • SmartBear_Suppo's avatar
    SmartBear_Suppo
    Icon for Alumni rankAlumni
    12 years ago
    Hi Amruta,

    I'm terribly sorry for the slow response here. This does look a bit odd.

    If you could provide some reproducible way for us to test this, that would really aid our investigation.

    Regards,
    Arian
    SmartBear Sweden