Video Interview | Michael Punsky - Performance Testing for UI Testers

Hello Community,

 

I'm happy to share with you the May SmartBear Community Interview. Today, we will talk about the importance of performance testing with Michael Punsky. Michael will tell us:

  • what performance testing is,
  • what the difference is between performance testing and functional testing,
  • what to pay attention to when creating your first performance test,
  • specifics of datasets for load tests,
  • and much more.

 

About Michael

Michael Punsky, President, and Chief Technologist at FTL Metrics Inc. - https://ftlmetrics.com/

Michael is a true performance testing expert with 20+ years of expertise.

 

Watch the interview here:

 

 

Text version:

Tanya

Today, I'm happy to welcome our speaker Michael Punsky. Hey Mike!

 

Michael

Hey! How are you?

 

Tanya

I'm good thanks! How are you?

 

Michael

Great!

 

Tanya

Thanks for joining us today. And, you're connecting from Miami, Florida, right?

 

Michael

A little bit north of Miami, about 40 miles, but close.

 

Tanya

The Sunshine State, I'm jealous! Tell us a bit about your background before we dive into the world of performance testing.

 

Michael

So, my background is - I was in the Air Force for 12 years. While I was in, I was in computer systems and communications, and then, when I got out of the Air Force, I transitioned into doing the same job but as a civilian. That was in 1994, so, I have been working with computer systems for a very long time. And, for a good portion of that time, probably, since 1997 or 98, also, I've been working with load testing software.

 

Tanya

Great background! So, you started many years ago.

 

Michael

Mm-hmm. Everything has changed a lot in that time.

 

Tanya

Oh, that'll be interesting to learn today. In the SmartBear Community, we have many users who are doing functional testing, API testing and many of them are asking what performance testing is and why it is important nowadays.

 

Michael

Well, I think that people always say, ‘What’s the difference between performance testing and functional testing?’ And, really, I think that there are a lot of similarities, but, in addition, there are some slight differences. So, in a functional test, you may have one process or one user that is stepping through and making sure that everything works the way the specification says it's supposed to work. If you type in these numbers and push this button, the result that you will get is this number. With performance testing, it's a little bit different. You say, ‘What if two thousand people enter numbers and push the button at the same time?’ Or, ‘What if they're all doing different things at the same time?’ So, I think what happens a lot of the time is development teams think about their functional testing because they have to make sure that what they wrote works the way that it's supposed to. But, sometimes, it's an afterthought to think, ok, now, what happens when a lot of people are in the application working, doing different things? Is it still going to work? And, is it going to return the results in a timely manner? So, in that way, it's a little bit different. In another way, it's the same because in a functional test, you're making sure that you get the result you're looking for and in performance testing, you have the same thing because a lot of times when you have a high number of people using the system, you are going to experience errors. It could be a problem with concurrency. A number above three or four hundred doing the same thing at the same time, there's a chance that two processes will be doing the same exact thing at the exact same millisecond and cause a problem in the system. Or, under a heavy load, things might start to break down, and then you'll just start getting server error 500, which basically says, ‘Hey, you sent something for me to process, I don't know what it is, and I don't know what to do with it.’ Boom! And then, it throws back that error and says, ‘I don't know what's wrong, but something's definitely wrong.’

 

Tanya

So, nowadays, we do everything fast, right? The speed is very important. For example, if I open a web page and if this page is being loaded for, say, 10 seconds, I'll simply close this page. I won't wait until this page is fully loaded. At the same time, the vendor of this page, they would like to avoid the situation. Is this something that performance testing can help me with?

 

Michael

Exactly. As a matter of fact, I have a customer, who shall remain nameless, and I've been doing some testing for them for the last two weeks. When I first checked to see what the speed was on their site, just bringing up the home page took 12 seconds with no load on the system. Just one virtual user testing their system, and it was 12 seconds. Which is totally totally unacceptable. So, over the course of the last two weeks, we've run a number of tests, we've found some actionable items that they could go back and remediate so that their performance improved, and then we test it again.

So, one thing to say about a performance testing strategy is that you're going to test, you're going to verify the results, and give the developers, or it might even be another group, it might be your network people or somebody else in that chain, the ability to make some changes, to improve things. And then, you're going to test again, get the results, and find out, you know, whether you've improved, how much you've improved, and what is left to fix.

So, in the course of the last two weeks they started at 12 seconds. That was on Tuesday last week. We tested again on Thursday, and, now, they were down to four seconds because of the changes that we made. And then, we made some additional changes, they made additional changes yesterday, and when I tested them last night, we were averaging under one second per page with 400 users on their system at the same time.

 

Tanya

Were the issues related with the hardware of their server? Or, was it something on the development side?

 

Michael

Okay. None of it was hardware. So, that's a good answer because hardware sometimes can be an expensive fix. It was software-related and settings-related.

 

Tanya

And, if we are talking about the amount of people just to test the load of the website, I know that for functional testing for example it is possible to create a dataset where the tests can use different data, for example, names, phone numbers, etc. are the requirements the same for load testing? I mean creating these datasets, and if they are, are there any specifics of the data generations for load testing?

 

Michael

It's the same. And, I think it's even more important with performance testing to use different credentials for virtual users because if you are using the same user, sometimes, you'll have a conflict. There are applications that only allow a user, an individual, to be logged in one time simultaneously. If I log in from my PC here in my office, and then log in from another machine, then it's going to say that there's a security violation because you can be logged in once. That's one type of reason.

Another reason is if you're using the same user and password, then you're going to be caching some things, the server will cache some things, the database will cache some things. And, databases, if you're calling for the same things again, and again, and again, are going to keep that information right at the very top available to give to you at a moment's notice, and it's not going to be doing the queries to bring things back. It's going to be handing you the same thing. So, there's that reason.

Additionally, you may want to have different types of users. So, user 001 is an administrator, user 002 is a purchaser, 003 is somebody that just runs reports. And, you're going to want to log in as each of those types of users, and you'll want to run them simultaneously so that you can see what happens if somebody is running a report. And, I've had this happen before, too. You could be running 2,000 people on the site, and they're all having a good time, and everything's coming back and performing great. And then, one guy runs a report, and the entire system crawls to a halt because that report is making so many queries to the database, and, you know, chances are some of those queries have not been optimized, need to re-index or use a different index. So, there are many reasons, but, yes, you do want to use different parameters in your test. User ID and password is one, search terms might be another, just as an example.

 

Tanya

Okay, I already love performance tests! And, it looks like it can resolve all my issues. For people, who decided to move to performance testing, who decided to start doing it, what would your recommendations be on where to start? Their first steps, the things they will need to pay attention to?

 

Michael

I think if somebody is doing performance testing for the first time, I would suggest doing a little bit of reading and finding out some of the knowledge that's out there in the load testing community about what you should test and how you should test it, looking at some YouTube videos. There are YouTube videos that SmartBear has that tell you exactly how you should go about certain tests. I've even recorded some that are out there that talk about using LoadNinja and using the different features within it, but there's lots of information out there.

Another thing you could do is for your first set of load testing maybe call on an expert, have an expert come in, let them run the load test and share what they did, how they did it, so that you can do it yourself going forward. I know when I go out and run a performance test for customers, I like to give them as much knowledge as possible so that if they do want to do that themselves they can do it going forward.

 

Tanya

Thank you, that was very interesting! As an expert, just a short question - what future should we expect for performance testing?

 

Michael

I think the future is going to be very interesting in performance testing. I think AI is going to be something that is going to be used in a lot of load testing going forward. And, there are different ways that AI can be used in performance testing. So, it will be interesting to see what actually happens. One of the things that can be done is the AI could run the tests, could see what results are coming back, and then modify the testing in such a way to get even better results from your system, and then be able to give you recommendations, or, you know, a whole list of recommendations on ways that you can improve the performance of your site.

 

Tanya

That's interesting. Somehow analyze the results of performance testing and think of how to improve it. That's interesting.

 

Michael

Exactly.

 

Tanya

Great. Thanks a lot that was very interesting and useful, and I learned lot about performance testing, I'll try to do something with my Community website. Thanks a lot, thanks for your time, thanks for the interview.

 

Michael

Ok, thank you, Tatyana. Bye everybody!

 

Tanya

Bye!

 

Community Manager
Users online (118)
New Here?
Join us and watch the welcome video: