Walking Randomly

There are many ways to benchmark an Android device but the one I have always been most interested in is the Linpack for android benchmark by GreeneComputing.  The Linpack benchmarks have been used for many years by supercomputer builders to compare computational muscle and they form the basis of the Top 500 list of supercomputers.

Linpack measures how quickly a machine can solve a dense n by n system of linear equations which is a common task in scientific and engineering applications.  The results of the benchmark are measured in flops which stands for floating point operations per second.  A typical desktop PC might acheive around 50 gigaflops (50 billion flops) whereas the most powerful PCs on Earth are measured in terms of petaflops (Quadrillions of flops) with the current champion weighing in at 16 petaflops, that’s 16,000,000,000,000,000 floating point operations per second–which is a lot!

Acording to the Android Linpack benchmark, my Samsung Galaxy S2 is capable of 85 megaflops which is pretty powerful compared to supercomputers of bygone eras but rather weedy by today’s standards.  It turns out, however, that the Linpack for Android app is under-reporting what your phone is really capable of.  As the authors say ‘This test is more a reflection of the state of the Android Dalvik Virtual Machine than of the floating point performance of the underlying processor.’  It’s a nice way of comparing the speed of two phones, or different firmwares on the same phone, but does not measure the true performance potential of your device.Put another way, it’s like measuring how hard you can punch while wearing huge, soft boxing gloves.

Rahul Garg, a PhD. student at McGill University, thought that it was high time to take the gloves off!

rgbench – a true high performance benchmark for android devices

Rahul has written a new benchmark app called RgbenchMM that aims to more accurately reflect the power of modern Android devices.  It performs a different calculation to Linpack in that it meaures the speed of matrix-matrix multiplication, another common operation in sicentific computing.

The benchmark was written using the NDK (Native Development Kit) which means that it runs directly on the device rather than on the Java Virtual Machine, thus avoiding Java overheads.  Furthermore, Rahul has used HPC tricks such as tiling and loop unrolling to squeeze out the very last drop of performance from your phone’s processor . The code tests about 50 different variations and the performance of the best version found for your device is then displayed.

When I ran the app on my Samsung Galaxy S2 I noted that it takes rather longer than Linpack for Android to execute – several minutes in fact – which is probably due to the large number of variations its trying out to see which is the best.  I received the following results

• 1 thread: 389 Mflops
• 2 threads: 960 Mflops
• 4 threads: 867.0 Mflops

Since my phone has a dual core processor, I expected performance to be best for 2 threads and that’s exactly what I got. Almost a Gigaflop on a mobile phone is not bad going at all! For comparison, I get around 85 Mflops on Linpack for Android.  Give it a try and see how your device compares.

Links

• RgbenchMM on GooglePlay
• Prelim Analysis of RgbenchMM – Some of the in-depth details of the benchmark, written by the app’s author.
• Supercomputers vs mobile phones