Using webpagetest and nanovisor.js to Measure Web Performanc

1. USING WEBPAGETEST AND NANOVISOR.JS TO MEASURE WEB PERFORMANCE AND QUALITY IN TERMS OF USER EXPERIENCE BY KARAN KUMAR AND GIRISH VAITHEESWARAN

2. The web of 10 years ago was massively simpler with a narrow set of browsers and devices. These days the complexity of the devices, websites, and web applications has increased dramatically. As you can imagine, that makes for a challenge in measuring the performance of web delivery systems. Now add in the very different approach Instart Logic takes to deliver the web, using technologies like big data and client-side virtualization, and the measurement challenge grows again. As a result, we had to come up with a new type of framework for measuring web performance.

3. Two unique things about our Web Application Streaming approach are the dual-sided architecture and the way the Instart Logic platform sends information to the browser – we stream objects, rather than downloading the full objects the way that most web content has traditionally been delivered. EXISTING FRAMEWORKS TEND TO SIMULATE CLIENTS, RATHER THAN MEASURE ON REAL BROWSERS We found that many existing performance frameworks are tailored to measure web performance characteristics on the server side and simulate clients rather than use real browsers. The dual-sided architecture of Instart Logic includes a client-side virtualization component built in JavaScript called NanoVisor.js that resides in the browser and demands a special-purpose performance framework that allows real browsers and devices to be used. The second unique aspect is our streaming technology, which is enabled by the dual-sided architecture. Most performance and testing frameworks are designed to measure the download time of full objects on a web page. This metric does not accurately model the performance characteristics of streaming web application objects and it fails to credit the earlier time to interaction with the content which web application streaming makes possible. So those were a few of the challenges. Now let’s talk a little bit about how we do it.

4. A FRAMEWORK FOR MEASURING WEB PERFORMANCE THAT LOOKS AT IT FROM THE END USER PERSPECTIVE At Instart Logic, as computer scientists and performance geeks with years of experience in big data, large-scale distributed systems, and even traditional web delivery systems, we always look to deliver a faster web experience based on data-driven decisions. To this end, we have architected a performance framework called Phoenix that is built on top of our favorite open source web performance testing system, WebPagetest.org (WPT), of which we have a big internal instance set up. Phoenix uses our client-side virtualization library NanoVisor.js, which is a hypervisor for web applications in the browser. We leverage the core capabilities of WPT around instrumenting to provide a ton of detailed information from real browsers running on real devices. We couple all that information with some very low-level data which our NanoVisor.js library can collect as web pages load up in a browser. Around the core capabilities of those systems we have built a big data solution to understand and analyze the large amounts of data that our system generates. With such a framework in place, we can point to any website or web application and gather relevant data to understand its performance in depth. Similarly we can run any such application with Instart Logic’s network, thus giving us a comparative and competitive view. We routinely pull many thousands of websites and applications through the Instart Logic Web Application Streaming Network – and compare it to the performance without our service – to measure quality and performance from the standpoint of the end user.

5. We define such views by various “observation points,” with some examples below: • Platform: Desktop vs. Mobile vs. Tablet • Devices: Samsung Galaxy S4 vs. iPhone 5s vs. iPad 2 • Browsers: Chrome vs. Firefox vs. Safari vs. IE • Network: 3G, 4G, LTE, Cable, FIOS • Protocol: HTTP vs. HTTPS MEASURING PERFORMANCE GENERATES BIG DATA AND ALLOWS MORE INSIGHT By collecting and measuring information from the instrumented browsers via WPT and then also capturing detailed client-side performance data reported by the NanoVisor.js, we get invaluable insights into the user experience across devices, platforms and networks beyond what you can get from the basics provided by a browser’s built-in performance APIs. Such amounts of raw data from these many different observation points are collected in a big data solution we designed internally to achieve deeper insights. With the Instart Logic core team coming from the big data world this part was actually the easiest to tackle. Not only is this system invaluable for testing the performance and quality of our systems, but we have already leveraged it to provide deeper insights into how the modern web is evolving.

6. Here are a few examples. • We track changes in major trends on the web such as changes to when resources are being loaded such as before or after onload, via static references in HTML or dynamic references in JavaScript. • We constantly provide the Instart Logic R&D teams with feedback on existing product features and how they impact performance across a broad set of sites. • The data we collect also enables Instart Logic R&D teams to estimate how impactful a new feature idea would be. For example we might want to know how many SaaS apps are using multiple versions of common JavaScript libraries, and whether those libraries loaded up in the header or down in the page body. • That’s just a quick overview of what we have built and some of the things we can do with our performance and testing system. In future posts we will drill down in a few areas and also share some of the insights we have learned. • We wanted to close with a big thank you to Patrick Meenan and the web performance community for WebPagetest.org, which laid the groundwork upon which we build!

7. Visit our Blog to learn more