Background

Many large corporations are moving their mission critical applications to the Cloud. Some are in the progress of automating deployments while doing so, and automate testing as well. These activities represent massive changes in how the applications are developed, deployed, managed, and tested.

At the same time, and possibly due to these changes, traditional performance testing is being challenged. Not only are traditional performance testing techniques hard to automate from a Continuous Deployment perspective, the nature of performance testing itself is questioned. And, I believe, for the right reasons, as well. More and more web applications are Javascript-heavy, using frameworks like Angular and JQuery, posing real challenges for the CPU and memory of browsers, especially mobile ones. In addition, Front End frameworks and uncareful Front End programming can easily lead to intense network traffic, due to inefficient AJAX calls and improper caching of statics, application, and data. This high, often very latency sensitive  network traffic is a specially detrimental for applications running over delicate mobile networks, causing slow loading of web pages, crashes, and unexpected application behavior, when the network quality decreases, e.g. due to telecom tower switching in cars or trains.

The challenges of traditional load testing

Traditional load testing usually is not done using a browser. In stead, the most popular load test tools, also the commercial ones, try to simulate the network traffic that browsers produce, but do not use a browser for that. That poses a number of disadvantages.

Firstly. the reported response time does not cover the browser based response time contribution.  As we saw above, we might, and we will, miss a substantial part of the response time.

Secondly, even though network simulation capabilities is usually a feature in load testing tools, this simulation hardly ever represent how the browser reacts to slower networks. This is due to the fact that the behavior of browsers handling connections, caches and SSL encryption, to name just a few, is very hard to simulate.  That means, slow networks are hard to simulate using traditional load test tooling: this is probably the main reason, why these features of the load test tool are hardly ever used.

Thirdly, since the traditional load test tooling do not use a browser, they cannot determine the difference between the different browsers, or between different browser versions. With the move of more and more logic to the browser, small differences in how the browsers deal with code can have a high impact on the load time of the page: An important reason why some browsers are very popular, as they “feel” fast, while others are more and more unpopular, as they are perceived as unbearably slow.

Finally, modern web pages include a dynamic build up of the web page, to show the most meaningful information as soon as possible to the end user, while other, less meaningful information, is shown later: the time to the first meaningful paintis an important denotation of how fast a site feels to the end-user. Only a real browser can show the first meaningful paint: traditional load testing tools that only simulate the network traffic of the browser cannot and will not.

Moving load testing to the Cloud

You could wonder, if load testing using a real browser is so much more production-like, why has is not been done before? Well, because of many reasons, but one of the main ones has to do with the fact that if you want to use a real browser to simulate an end user, then you need many browsers. And they use a lot of CPU and memory, that was not available until now. Could solutions, whether extern with a well-known Cloud provider like Microsoft Azure, AWS, Pivotal Cloud Foundry or Google Cloud, or intern in the company using a private-version of the before or using Kubernetes, provide this kind of processing power, especially if used only during the load test, and freed after the load test. So, in a sense, Front-End testing and scalable Cloud solutions go hand in hand.

Introducing browser-based load tests

So, combining Browsers in a Cloud solution is the way to go, in fact: I see it as the future of load testing. They provide the much needed real user experience, open the way for shared scripting efforts of functional and load testers, and simplify load testing complex Front End web applications. How to achieve this?

There are a number of tool vendors that now include Selenium integration in their load testing solutions. In fact, some vendors like Microfocus, offer a commercial alternative to Selenium specialized in performance testing the end user experience in the browser, called TruClient. However, TruClient was rarely employed to produce high load, as it requires a lot of hardware resources. Instead, during load testing , TruClient was traditionally used with one or two instances, to measure the end users view on performance, while protocol based load testing tools produced the bulk of the load. Now, with Cloud scalability, browser based tooling can produce the load as well, eliminating the  need for both a protocol-based and a browser-based script.

Indeed, there are freeware solutions out there as well, that, with some experimenting, can be put to use. JMETER has a Selenium Webdriver sampler that can be used in a Selenium GRID configuration. Combined with a Cloud solution, the Selenium Grid can scale almost unrestricted, only being limited by the available underlying hardware.

Conclusion and a recommendation

For performance testers, it is now a smart move to start to learn about browser based testing techniques. The future of performance testing does not lie in more and more protocol based testing. Even though we will probably always need protocol based testing because we will need to load test APIs as well as browsers, now is the time to find an introductory guide on how to use Selenium. And experiment with Selenium-based response time measurements for a Javascript-heavy application. And as you may find, slow front-end websites are abundantly available out there to try your script on.

About the author

Arthur Luimes is a Senior Performance Consultant at Altersis Performance, with more than twenty five years of experience in performance management, and is leading the way to embrace new technologies to enable performance optimization.