- For a while now we've been keeping track of mobile browser performance using two relatively popular JavaScript heavy benchmarks that are a regular fixation in our smartphone reviews. If you've read any of those reviews, you should immediately be able to name them - SunSpider and RightWare's BrowserMark. Tracking JavaScript performance thus far has helped us codify and track SoC performance, but really understanding and quantifying overall browsing smoothness has remained a somewhat more daunting task. Moreover, JavaScript performance is just one facet of many that contribute to the user's perception of browsing speed.
Left to right: T-Mobile SGS2, Motorola Atrix, AT&T SGS2, HTC Sensation, LG Optimus 3D, Xiaomi Mi-One
Smartphones | ||||||||||
T-Mobile SGS2 | Motorola Atrix | AT&T SGS2 | HTC Sensation | LG Optimus 3D | Xiaomi Mi-One | |||||
SoC | 1.5 GHz APQ8060 | 1.0 GHz Tegra 2 | 1.2 GHz Exynos 4210 | 1.2 GHz MSM8260 | 1.0 GHz OMAP4430 | 1.5 GHz MSM8260 | ||||
RAM | 1 GB LPDDR2 | 1 GB LPDDR2 | 1 GB LPDDR2 | 768 MB LPDDR2 | 512 MB LPDDR2 | 1 GB LPDDR2 | ||||
OS | Android 2.3 | Android 2.3 | Android 2.3 | Android 2.3 | Android 2.2 | Android 2.3 |
If you've been reading our reviews, basically every device except for the T-Mobile SGS2 (which we're going to review shortly) and the Xiaomi Mi-One should be relatively familiar. The Mi-One is from a Chinese OEM we haven't seen devices from before called Xiaomi, but is a surprisingly decent performer and has one of the most unique (and surprisngly speedy) skins I've ever seen on Android.
Vellamo Test Breakdown
Vellamo's test are divided into a few categories, and start with Rendering tests which stress the performance of HTML5First among these tests is See the Sun Canvas, which rendes increasingly demanding stages of a 2D rotating butterfly in an HTML5
The Xiaomi Mi-One is a new device for us, but delivers strikingly fast
performance thanks to the work that Xiaomi has done to the stock Android
browser. I found the SGS2's backing store pretty decent as well, but
you can see how much difference there is in performance between the two
despite packing the same 1.5 GHz dual core snapdragon (APQ8060 and
MSM8260 differ solely in baseband - APQ lacks one).
The next test is Deep Sea Canvas which tests almost the same parameters
as See the Sun, except tests are non-sequential here and are an attempt
to fully load the platform.
Interestingly enough, Exynos 4210 comes out on top of the pack.
Next up is Aquarium canvas which tests the number of HTML5 canvas
objects that can be rendered simultaneously. In addition, this test runs
in a native Android Java implementation for comparison with the HTML5
Vellamo calculates a score for each subtest alongside FPS and dropped
frames, and we've chosen to report FPS wherever possible, but this test
makes more sense being reported in base score. Interestingly enough the
Sensation 4G edges out the faster T-Mobile SGS2.
Pixel Blender is next which draws multiple images over a static
background and applies alpha blending. Vellamo calculates the graphics
throughput of the browser in pixels per second to arrive at memory
bandwidth for blitting and blending.
Again there's a pretty shocking disparity between Samsung's browser and
the Xiaomi, we're reporting the base score here as well.
Next up are the very familiar JavaScript VM tests which we've taken a
look at many times before in numerous reviews. In Android's case, every
browser is using the V8 JavaScript engine. First among these tests is
SunSpider 0.9.1. We started using 0.9 and have moved to 0.9.1 for most
things now, which still consists of a number of relatively short tests
run 10 times. The final product is a completion time in milliseconds,
and thusly lower is better. SunSpider is developed by the WebKit team of
course.
SunSpider is to WebKit as the V8 benchmark suite is to, well, V8. It's
another JavaScript benchmark which consists of a number of beefy tests
(raytracing, regex, math, e.t.c.) and also exercises the VM's garbage
collector. Vellamo runs v6 of the V8 benchmark which is latest.
Next is SurfWax Binder, which measures the efficiency of JavaScript
bindings code which links the VM to the rest of the browser's core and
rendering core. It consists of a long series of nested function calls,
which Vellamo measures in number of accesses per second.
There are three tests here, starting with Ocean Flinger, which measures user experience when scrolling a page in a flinging manner. This is a test which consists of sample text, images, CSS, and JavaScript. ImageFlinger is a similar test which takes a dense grid of repeated JPEG images and scrolls it. Finally Text Flinger does the same thing, but with a repeated grid of densely packed text. We've reported Ocean Flinger in other tests before since it really is a combination of images and text (like we've already stated).
I have to admit that Vellamo's numbers here closely mirror my own
subjective scrolling performance perceptions when using the respective
devices. The three scrolling tests represent a way to finally codify one
of the more elusive (and yet easily perceptible) browser performance
aspects.
The final test is a networking loader suite which loads a page stored
locally and times how long it takes in milliseconds. This is done in
both hot and cold mode (with the caches cleared and not cleared) and is
designed to test local performance.
We're going to be adding (and simultaneously giving the same treatment) a few more benchmarks in preparation for a revamped testing suite for 2012 and an updated smartphone Bench as well.
No comments:
Post a Comment