Measuring Mobile Web Performance: 101
by Peter
Short version:
From the end users perspective on a real device, connected to a Carrier network, anywhere in the world.
Long version:
Well here things get a little more complicated. There are always two different ways to measure User-Agent (the device) performance, whether it’s on the desktop or Over the Air (OTA)/On the device. This translates into two different “categories” of performance testing.
Over the wire:
The first is simply testing to see what happens with any particular application “on the wire”. This includes all the simple “packet capture” style tools that are out there. These kind of test, historically, may or may not help you solve performance problems being seen with any particular User Agent (the device). Just watching some downloads take place doesn’t really tell the “whole story”… especially when it comes to analyzing something as complex as a modern HTTP protocol exchange.
Presentation/Application layer:
The second is testing the presentation/application layer itself. Whilst it’s important to know how efficiently the transport layer is functioning… in terms of sheer PERFORMANCE… it is just as important to know how quickly the resources are actually becoming available for use to the application itself, or what JavaScript errors might be taking place during the page loading, or a host of application level “conditions” that contribute to overall (User Perceived) performance
Before you can measure anything on mobile you have to understand the fundamental differences between two categories.
Finally – Any testing MUST include testing the actual Carrier performance on a real device which a user would hold in his or her hands. There’s lots of “back end” fake Mobile sites around – sorry these don’t count as the real user experience. Carrier network performance is NOT the same as a copper connection to the Internet at your desktop. It’s something entirely different.
Finally one more critical item to consider – the devices real time location. Just as the Carrier network effects performance so does the location of the actual device.
Ignoring any of these conditions means that you’re not measuring Mobile performance.
Posted in: #webperf, Context, Performance, User Experience