Forum Discussion

vondie's avatar
vondie
Contributor
7 years ago
Solved

Difference between Test Case Level and Test Step Level

I am new to load testing and trying to understand the Test Step Metrics in a test case with only one test step. What is the difference between Test Case Level and Test Step Level in regards to the Median? The Median values are five seconds apart... it seems like they should be the same though if there's only one test step. The same with errors. Why are they showing different values?

 

  • Hello Vondie, 

     

    I recommend using the Statistics tab for these details, which uses a different way of mapping the results. It should not display this same issue. The real-time chart and statistics on the "Load" tab is using a different method of collecting data and more details below.

     

    Basically it comes down to how this report is created. It is trying to capture events happening in a specific second and for some reason there is a separation of the test case and test step registering in the same time window on the report. I've created a research exercise for engineering that should kick off a longer-term effort to make this chart more clear. They should be showing the same values in your test setup and they do not. 

     

    More to come on this in the future and thank you for posting this.

2 Replies

  • JamesLunan's avatar
    JamesLunan
    SmartBear Alumni (Retired)

    Hello Vondie, 

     

    I recommend using the Statistics tab for these details, which uses a different way of mapping the results. It should not display this same issue. The real-time chart and statistics on the "Load" tab is using a different method of collecting data and more details below.

     

    Basically it comes down to how this report is created. It is trying to capture events happening in a specific second and for some reason there is a separation of the test case and test step registering in the same time window on the report. I've created a research exercise for engineering that should kick off a longer-term effort to make this chart more clear. They should be showing the same values in your test setup and they do not. 

     

    More to come on this in the future and thank you for posting this.

    • vondie's avatar
      vondie
      Contributor

      Thanks, James! I am looking forward to seeing this enhanced!