Forum Discussion

RaniS's avatar
RaniS
New Contributor
9 years ago

Need clarification on TPS/Count/Last/Median..

Hi Team,

  I'm new to LOAD UI NG, i'm trying the sample project Baseline Load Test.  Please clarify the queries in the observation below.

 

Here's my config's:

Load profile – Fixed and Load type – Vus : 1 Vus for 10 sec

Only a single city search :- Beijing

 

Observations:

Test Step Metrics

1. Count starts at 0, the value of count doesnt match with the no. of request simulated as per Global metrics. Pls advice how this will help to relate when we are load testing with large no.s [ I feel the issue is when the 9th or 10th(last) request is submitted, the pop for Preparing reports displayed so the background count is not updated properly]

2. Last returned value shows up 15, i guess it's the 'time taken' value on 10th sec.

3. Need help with the calculation on Median for Test Step level and Test case level.

 

Global Metrics:

1. How the TPS value is calculated and plotted in the graph?  [0->2 - 5 - 6 - 5 - 5 - 6 - 6 - 5 - 5 - 5]

 

 

 

 

  • TanyaYatskovska's avatar
    TanyaYatskovska
    SmartBear Alumni (Retired)
    9 years ago

    Hi Ranis,

     

    Let me answer your question:

    #1

    Count shows how many times the scenario was simulated. There is a case that the scenario started executed, but the execution time for all scenarios (you specified) has finished. It means that the request was sent to a service. However, the tool doesn’t get the response as the time was over.

    This can explain the difference between the number of the requests and the number of simulated scenarios.

     

    #2

    Last shows the time of how long your scenario was simulated last time.

     

    #3

    Median time shows the middle time of all scenarios sorted in ascending order. Please refer to this Wiki article for more information.

     

    Also, I recommend that you read the following Ready! API article describing the information shown on the Load page:

    http://readyapi.smartbear.com/loadui_ng/results/load/start

    • RaniS's avatar
      RaniS
      New Contributor
      9 years ago

      Thanks Tanya for the clarification. But still i see an issue with the count. I activated the trail version to verify more details. Same sceanrio i ran with the trail. Following are the observations:

       

      This time the count was 10 as expected.

      #1: Transaction log lists 10 requests with their response times but Test Step History in the Report lists from 2nd to 9th entries of transaction log, with 10th request is updated as NA.[ Fig 1 and Fig 2 attached]

       

      #2: Global Mertics graph for Min values - At 1sec, Min value was 0 (which is not matching with Test step history report values [NA]) and at 10th sec, the value was 5 (which is again not matching with Test step history report values [NA]). [ Fig 1 and Fig 2 attached]

       

      #3:Global Metrics graph for Max values - At 10th sec, Max value was 24 (which is not matching with Test step history report values [NA]) [ Fig 1 and Fig 2 attached]

      [Note: 9th sec value is extended to 10sec in both the cases]

       

      #4:TPS in 'Test Step Metrics - GeoCode Search' was 0, Please clarify if the value was taken from 10th sec of TPS value which is 0 in Test step history report? 

       

      #5: TPS in 'Test Step Metrics -Test Case level ' was 0, shouldn't it be the average TPS for the total test duration?

       

      #6: Global Metrics for TPS Graph :- Not sure how the TPS was plotted for each sec. From the test step history report, TPS value at 1st sec - 9th sec was 1 and for10th sec the value was 0. But the TPS plotted in the graph are as follows [2. 4. 3. 5. 6. 6. 6. 5.5. 6] [Fig 3 attached]

       

      #7: Need clarification on the mismatch between Graph plotted (Min/max), Transaction log and Test step history report. [Fig 6 attached]