Does cloud eliminate the need for performance testing? Short answer: ‘No’!

For years we've dealt with IT professionals and organisations who have responded to software performance issues by purchasing more servers. Today that same thinking leads many to assume that performance issues with cloud hosted applications can easily be resolved with the elastic scalability of capacity. So it is natural to ask, does cloud solve performance issues and as such 'does cloud eliminate the need for performance testing?'

Short answer: 'No'

Long answer

When we look at performance issues at Wild Strait we consider three areas:

• Stability
• Capacity
• Response time

In our experience cloud only helps resolve a very small number of the issues we find in these areas.

Capacity related performance issues may be resolved with cloud

Where an application is not performing due to insufficient capacity then the elastic and scalable capacity of cloud may help with performance. However, in our experience...

Most performance issues are not capacity related

If we go back to the old days, we found most often that performance issues continued even after the IT professionals and organisations had purchased and implemented the additional server capacity intended to resolve a performance issue. The same applies today with cloud. In other words, regardless of whether it is a server application or cloud hosted, by far the majority of performance issues that we see are not capacity related, so increasing capacity often does not resolve the performance issue.

Instead we find that most performance issues relate to software and software configuration issues such as memory leaks, database locks, missing database indexes or misconfigured settings (such as message size limits or frequencies). Take the example of a misconfigured message setting, the application might work well when it sends messages one at a time, but regardless of cloud capacity, the application may encounter performance issues when it sends loads of thousands of messages or batches messages together.

As I prepared this post a couple of our senior Wild Strait performance testing specialists chipped in with these comments on this topic:

• Stephen: "Almost every performance issue I've ever encountered was related to software, and scalable hardware may not significantly improve performance in these instances."
• Ben: "100% agree that most performance issues are not about “the kit” and cannot be solved simply by throwing more tin at it."

Response time may be hindered by internet latency

Cloud-hosted applications are often used via the internet. Internet latency may introduce risks to response time, especially for chatty applications and applications that need to interface with other components over the internet

Even if your cloud hosted application performs well in one location (perhaps a location with some proximity to the cloud service provider's data centre) will it perform in all locations where it needs to be used such as New Zealand, Mali, the United Kingdom? Testing can help confirm this and increasing cloud capacity may not resolve any performance issues identified with this problem.

A poor performing application may invalidate your cloud SLA

Cloud service level agreements (SLAs) may guarantee a certain level of performance. For example, 0.5ms latency at 1000 I/O operation per second. However, if your application performance falls outside of the standard parameters as set out in your SLA, then this may invalidate the cloud service provider's performance guarantee. If you have not undertaken performance testing you may not know whether your application is within the required parameters or not.

Don't pay for cloud capacity that you do not require

While cloud can be a cost effective approach, increased capacity is not free. When you attempt to solve a software or configuration related performance issue by providing more capacity, you are inefficiently using more capacity than is required for that application. Why pay for more cloud capacity than you require? Similarly, be aware that your application and its data may grow over time, so understand that the cloud costs to your business will most likely increase in the future.

Performance test your cloud application

If there is a risk to your business if a particular application does not perform, then performance testing is most likely very important regardless of whether your application is to be deployed in the cloud or not.

We are always here to help, so please get in touch if you have any questions or concerns about the performance of your applications.