Anyone with a hand in operating a web application probably has at least passing familiarity with application performance monitoring (APM). And anyone with an interest in the application’s fate should have an interest. In the competitive software landscape, poor performance can sink your application.
Toward this end, savvy companies employ APM tools. With these at your disposal, you can learn an awful lot about your application’s production behavior and performance. Has the site gone down? Do certain pages take forever to load? Or is the application throwing an unusual number of errors? You can quickly learn these things plus plenty more.
Once you obtain this information, you probably do some root cause analysis. Perhaps a network node went down somewhere. Or maybe a rogue piece of hardware in some data center somewhere gave out, leaving you with a temporary problem. But you might also discover something more troubling. You might discover a complex root cause, and one deeply intertwined with the actual design of your application. And then you might have a serious problem.
We talk a lot about monitoring production apps, and rightfully so. This helps you discover all manner of problems, including both self-inflicted and external one. But when it comes to the self-inflicted kind, you need to catch those earlier. In addition to standard QA, you need a performance testing strategy. During all stages of development, you should constantly exercise your application in a variety of ways. So let’s take a look at the different kinds of application performance you should test.
Responsiveness to Users
Anyone worried about website performance, particularly as it pertains to SEO, has probably seen sites like YSlow. Should you allow your site’s page load times to become onerous, search engines actually begin to penalize you. So site owners find themselves motivated to carefully monitor page load time for users using tools like this.
Of course, the concern extends well beyond SEO. Slow-loading websites cause frustrated users to bounce. If you run a blog, this may frustrate you, but if you run an e-commerce operation, it may bury you. User experience of responsiveness matters, and it matters a lot.
And, as a matter of fact, it matters for every sort of application, web-based or not. As such, you should test for this routinely. Preferably you should automate this effort somewhere in your deployment pipeline. But even a concerted manual effort will help considerably. Make sure screens, pages, and forms load in reasonable amounts of time. Make sure the interface doesn’t hang or freeze during user operations. In general, make sure your application stays responsive at all times.
Next up, consider an area of performance testing known as load testing. Load testing refers somewhat vaguely to the volume of input applied to your application. You can think of it in terms of the load you ask your application to carry.
To understand the significance of this prong of your testing strategy, think of the responsiveness testing I just mentioned. If you put your web application into your pre-prod environment and have a QA person check page load times, you learn some interesting things. Sluggish response times might indicate poorly constructed database queries, bad server-side code, problems in client-side code, or even an internal network hiccup. But what does a snappy page load tell you, exactly?
To understand what I’m driving at, think about having a single QA person making requests of your web application. Does that reasonably simulate production? Or will you more likely have thousands or even millions of people banging away at your application? Load testing involves deliberately testing your application under the sorts of circumstances that you can expect to see in production. And you really need to do it. After all, you don’t want to find out whether your code works by shipping it to production and crossing your fingers. Do you really want to apply that strategy to see if it works at normal usage volume?
Hopefully, you’ve bought into the idea of load testing. Once you have that in place and squared away, you can think about stress testing.
In a sense, stress testing is simply a specific subset of load testing. You do a load test where you specify a load of “extremely large.” In fact, you want to keep throwing more and more load at your system in order to see at what point it cracks. You want to break stuff here.
Why do this? Well, it serves two important purposes. In the first place, it gives you actionable information about when you can expect problems. If you’ve never stressed your application, how would you know when to game-plan or take in the face of surges in load? Would you just wait for it to break? You need a way to say, “we’re currently at 90% of what we can handle, so we need to do something.”
The second reason speaks to continuously improving the application itself. When you stress it to breaking, you expose the weak link in the chain. In some cases, you may have no course of action. But, in others, you might be able to fix the weak link, making the chain stronger.
So far, we’ve covered the user’s perception of performance and how the system performs under varying levels of stress. For the final type of performance testing, I’ll cover today, let’s take a look at a different variable.
With endurance testing, you hold the load steady and examine performance as a function of time. To consider it from a pragmatic perspective, ask yourself the following. How confident would you feel in a load test that ran successfully for an hour? How about 5 minutes? Or 5 seconds? For some diminishing unit of time, you’ll wonder what you’ve really proven.
Endurance testing involves specifying a load to place on the system and then monitoring the system for an extended period of time. This helps you detect elusive issues that crop up only after hours or days of successive running, such as slow memory leaks.
Endurance testing often presents the most challenge of any form of performance testing. Sure you can run tests overnight in pre-prod, but sooner or later, you need to send the next version there. And, even if you set up a dedicated endurance testing environment, can you really simulate prod? Ideally, prod’s runtime should be forever. You’ll probably fall a bit short of that. But still, doing some endurance testing helps tremendously.
In this post, I’ve outlined some rather standard application performance testing strategies. I recommend that you adopt all of them if you haven’t already. But that doesn’t mean you should limit yourself, either.
Your application is going to do weird, unexpected things in the wild. The APM tools I mentioned early in the post will help you track these things down and analyze their root cause. But don’t content yourself with just fixing them. Devise an application performance test to add to your strategy so that the issue never crops up unexpectedly again.