It can be tricky figuring out which mobile device to buy. Some get longer battery life than the competition, while others offer more storage, memory, or special features like hands-free operation.
But some folks like to get the model with the fastest processor — and it turns out that may be a little harder to determine than you’d think.
That’s because while there are a series of apps that let you benchmark your phone or tablet’s performance, manufacturers have started to optimize their devices specifically to score well on those tests.
It’s sort of like teachers instructing kids about everything they need to know to pass a test. While it boosts the chances that they’ll ace the exam, it’s not clear if they’re learning anything that will help them in the real world.
Likewise, just because a phone scores well on the Antutu, GLBenchmark, or Quadrant benchmark doesn’t necessarily mean it’s going to be faster in everyday use than a model that achieved a lower score — because it’s not always transparent to the user just how that score was earned.
Some manufacturers have been accused of running fake, or modified benchmarking software in order to inflate their scores.
Now the folks at AnandTech have discovered that Samsung takes a different approach: The Samsung Galaxy S4 smartphone runs the CPU and GPU at full blast whenever you’re running common benchmarking apps.
When you close the app, odds are that your hardware will run more slowly. The result is that the score doesn’t provide a realistic indicator of real-world performance and if you try to compare your score with that of another device, you won’t really know for certain why one is the fastest.
You can find more details at AnandTech, but in a nutshell, there are two versions of the Galaxy S4. Whether you have a model with a Samsung Exynos 5 Octa processor or a Qualcomm Snapdragon 600 chip, the CPU will run at its highest speed while a benchmarking app is open.
With the Exynos model, the GPU will also run at 532 MHz. What’s strange about that is that while the graphics processor can handle that speed, the only time it ever seems to go that high is when you’re using benchmarking apps. When you’re playing games (which tend to be among the most resource intensive apps), it tends to top out at 480 MHz.
In other words – theoretically the Galaxy S4 is capable of performing just as well in other apps as it does in benchmarks. It just probably won’t.
Update: Samsung has responded, saying that it isn’t that the company’s software is tweaked just to perform well on benchmarks — it’s that it lowers the maximum GPU frequency when running some apps like games for a prolonged period to reduce the risk of overloading and damaging the system.
That’s actually a pretty reasonable explanation — although it doesn’t account for the fact that AnandTech found specific references to a “BenchmarkBooster” and a series of benchmark apps that were listed by name in the code of a Samsung app on the Galaxy S4.
via SamMobile
And thus the problem with sites always listing all these benchmarks instead of loading a cpu and/or gpu intensive item and comparing products side by side running that same item (game, video, browser etc.). . .
Benchmarks are synthetic anyway so there’s likely to be a difference between a benchmark score and real-world performance.
I’m sure next week we’ll hear about some clever game developer who has figured out how to fool the Galaxy 4 into thinking one of the relevant benchmarking apps is running every time his game is started up. You know, so that “Angry Birds: Hog Amongst the Pigeons” runs at top speed.
That’s a pretty smart idea! I also wouldn’t put it past the crowd at xda to come up a patch to enable the higher maximum GPU speed across the board.
Hahaha, Samsung’s response was just a flat out lie given that last paragraph.