API Load Testing Mistake #5 to Avoid: Not Factoring in External API Calls - Robert Schneider

TanyaYatskovska · ‎08-31-2020

Nowadays, lots of modern API-based applications make calls out to third-party APIs. These APIs can belong to your organization, or they can be located on a partner, customer, etc. side. Today, we will analyze the specifics of calling these APIs during API load testing. Robert Schneider, a software testing consultant from WiseClouds, will help us avoid the mistakes when working with external API calls.

Watch the video (find the transcript below) and post your comments:

Robert Schneider

Robert D. Schneider is a Silicon Valley–based technology consultant and author, and managing partner at WiseClouds – a provider of training and professional services for NoSQL technologies and high performance APIs. He has supplied distributed computing design/testing, database optimization, and other technical expertise to Global 500 corporations and government agencies around the world. Clients have included Amazon.com, JP Morgan Chase & Co, VISA, Pearson Education, S.W.I.F.T., and the governments of the United States, Brazil, Malaysia, Mexico, Australia, and the United Kingdom.

Robert has written eight books and numerous articles on database platforms and other complex topics such as cloud computing, graph databases, business intelligence, security, and microservices. He is also a frequent organizer and presenter at major international industry events. Robert blogs at rdschneider.com.

Also, Robert is an author of 7 Common API Testing Mistakes video series.

I hope you liked the video! If you have any questions, please post them here. This is a final video of the series. You can watch the previous ones here:

Subscribe to the SmartBear Community Matters Blog to get the news from us first!

Transcript:

Tanya:

Hey! I'm welcoming you in the ‘5 Common API Load Testing Mistakes’ series. We are doing it together with Robert Schneider from WiseClouds, and we have already discussed four major API load testing mistakes you need to avoid. And, today, we will cover the last mistake, but not the least!

As far as understand, this will be a mistake about using the third-party APIs.

So, Robert, the mic is yours! Tell us why this mistake is important, and why we need to avoid it!

Robert:

Sure! So, a lot of applications these days, a lot of modern API-based applications make calls out to third-party APIs. That API, that external API, might be one inside your organization, it might be one from a partner, or a customer, or a government entity, whatever it might be, and a lot of times what people are doing is - when they have their functional tests, if they write functional tests for their APIs, they stub out those calls. They never bother to make the call to the external API, because, quite possible, the third party doesn't have a sandbox. They don't have a test bed for you to work with. And, in your test environment, you can't call their production environment over and over again. Sometimes, there’s even a monetary cost to doing it that way. So, a lot of times what people will do is they'll simply omit the calls to the third-party external API as part of their functional testing.

And then, of course, when it gets to load testing where now you're going to start to bring up production levels of usage of your application it is understandable that people would not make the calls to those third-party APIs because you don't want to get the third party angry if it's a government, you certainly don't want the government angry at you. So, they simply bypass doing that, and that's really dangerous because you don't know what level of responsiveness you can get from that third-party API that you can expect in production.

Now, this is a mistake that people are making, and I'm going to give you some ideas about how you can surmount it. But, really, the only way you're truly going to know is to actually see it in production or look at the service level agreements that your third parties are giving you, but, for example, even under load, if you start to encounter delays on these third-party APIs, you might have timeouts that you didn't factor in, you didn't think about during the designing of your test. So, really, it's important to include those external API calls in the functional test that you're doing, a single user path through your application, as well as placing it under load and seeing how it can work.

Again, you don't have a lot of control over this. You can't go to the US government, or the EU government, or whatever government you might be dealing with, and say: ‘Hey, we're going to be sending you tens of thousands of transactions for testing!’ And, whether you like it or not, now, you can't do that, but what you can do is consider, maybe, using like a virtualization tool. Something like ServiceV from SmartBear under the ReadyAPI product. And say: ‘We're going to simulate what this third-party API is going to look like. We're going to simulate its functionality.’ Which you can do. Maybe, we'll even just have it do a straight echo back, who knows, but we're going to simulate that functionality as best we can. And, we're going to try to build in some latencies and see what happens. We're going to say, maybe, that external API that we're calling is going to have a 500-millisecond baseline that it's going to take for us to get results back. And then, we're going to build in some variability using the tool to simulate that.

Whatever way you go about doing this, it's important to make that effort. Because, again, as you might remember from the very first part of this series, if you're not testing a full functional business process, you really don't know how your application is going to behave at runtime when all the pieces are hooked together and operating. So, again, as our last in this series, the external APIs which are beyond your control, you still can do a lot with something like ReadyAPI and ServiceV to at least make a best guess at what kind of responsiveness you might get from those third-party APIs.

Tanya:

I know that some testers use some hard-coded data in case if they do not have access to the external APIs so your suggestion is to stop using it, right?

Robert:

Well, it's a start. It's certainly that you have to do something. So, some people will simulate this, and I've seen this many times firsthand. Some people will go to the developers and say: ‘Hey, can you stub out the calls on the back end to the third-party API?’ And, they’ll say: ‘Okay.’ First of all, that takes the developers off their main line of responsibility. Secondly, you're leaving certain parts of your application untested, so, really, you want to do the very best you can, and, I guess, I'm a huge believer in virtualization technology because it's exactly meant for this kind of scenario where you don't have to worry. Now, we're going to use some kind of virtualization technology to simulate that call to the government entity that we're talking to, and we know it's not perfect, but at least, we'll have some idea about what kind of latency we can expect or simulate latency, as well as you just said Tatyana, in terms of the data, you can use these virtualization tools to put some business logic on the simulated third-party API and actually start to see some behaviors in your application based on what you call them with. So, it really opens up the door for all sorts of much more sophisticated testing scenarios from a functional perspective, as well as getting it all the way down to the performance perspective again part of that life cycle of unit to functional, to functional under load testing that makes you so much better prepared to place your APIs into production.

Tanya:

Okay, great! Thanks a lot for all of your suggestions and for explaining the mistakes that we need to avoid when creating API load tests. I'm sure that our customers will use your suggestions and will create their best tests!

Robert:

Okay, well, thanks very much everybody! Thank you, Tatyana! I look forward to the next series of these discussions!

Tanya:

Sure, we will be happy to learn more from you.

Robert:

Thank you!