More often than not, companies have to play dirty in the race to the top. We have seen plenty of cases where a company was busted doing something really stupid just to get an edge over its competitor. Things like false advertisements, false specs by even established companies are not unheard of, but today let’s talk about something that Huawei was caught doing — benchmark cheating aka benchmark optimisation.
Huawei’s Benchmark Cheating Scandal
Huawei was recently caught by AnandTech optimising some of their smartphones like Huawei P20 to over perform in a specific benchmarking app, 3DMark. We have already talked about this here but the story in short is that Huawei P20 and a few other top flagships from the company were identifying benchmarking apps to extensively increase their performance with the primary goal of scoring way higher than in a traditional use case scenario.
In this age of AI, it’s not uncommon for smartphones to identify specific type of apps to boost performance. For example, new age phones can often detect gaming apps and switch to peak performance mode for the best user experience. However, what’s really the big issue here is that Huawei had tweaked its newer flagships to detect specific benchmarking apps (3DMark in this case) to run the phone at an unrealistic performance level. Apart from issues like overheating, such a level of performance decreases efficiency significantly, making the whole experience impractical for daily usage.
- RELATED: Our Honor Play Review Here!
Another issue with this type of performance optimisation is the motive. The big reason why companies like to optimise their smartphones for benchmark is to score higher than the competition. That’s not really a good enough reason to push your smartphones beyond its limits.
Our Obsession With Benchmark Scores
That being said, I can’t personally crucify Huawei for trying to score high on benchmarks. History has shown us that almost every manufacturer has been found tweaking their smartphones for better benchmark scores. This list includes the big names such as Samsung, LG, Asus, HTC, Meizu, and even OnePlus. So, the only reason why this was made such a big deal is because Huawei didn’t seem to get the memo that this was unacceptable, even in 2018.
Given the number of companies that have tried tweaking their scores for benchmark, the fundamental question is why they are doing this, even now.
The answer directly points to our needless obsession with benchmarks.
I get why benchmarks are so popular. It’s really hard to quantify the performance of a smartphone in any other way. Benchmarks give us a figure which we can compare to determine which one’s better than the other. It kind of makes sense. But then, it doesn’t. Because, getting this figure involves the app running a few resource intensive tasks to get a final score. The process isn’t fool proof and now you know that the results aren’t either.
We must start giving less importance to benchmarks and more importance to the phone’s actual user experience.
The Right Way To Evaluate A Phone’s Performance With A Benchmark App
Whilst a higher benchmark score in general doesn’t necessarily mean that the phone is better than others, users need to start understanding the scores in the right way.
Instead of obsessing over a particular score, we should start to evaluate a smartphone on the basis of levels of performance.
For example, take a very simple benchmarking app like AnTuTu. If a smartphone scores above a certain figure, for example 250K, its already established that the phone has a very powerful processor inside that can handle any task that you throw at it. Understanding that once a phone crosses a particular threshold, it doesn’t really matter whether the phone scores 290K or 261K on the app, is very important. What we need to realise is the simple fact that a score of 291K doesn’t necessarily mean that its better than a phone that scored 273K. In real life, the lower scoring phone could offer a much better experience than the other.
I know as enthusiasts, it’s really hard to ignore specific benchmark figures, but once we learn to stop this practice, we will start seeing the real picture. A SD835 smartphone with great optimisation is hundred times better than a SD845 phone with a bad optimisation. After all, would you be happy with a phone that scores 280K on AnTuTu but has a crappy UI or a phone which scores 240K on the same app, but handles everything with butter smoothness?
Let’s start giving less importance to individual benchmark figures and more importance to their real life performance and we would no longer have to worry companies optimising / cheating on these apps. After all, figures do lie!
I know, its easier said than done. It’s really tempting to see higher and better benchmark figures on flagship devices every year, but as consumers, if we start giving importance to the right things, in the long term the industry will benefit as a whole.
So the next time you see a company marketing a very high benchmark score for its flagship, just say — that’s pretty cool but how does it perform in real life?
