UPDATE: After the below episode, some of the most important Huawei handsets have been delisted from the following benchmarking platforms: UL Benchmarks, Futuremark and 3DMark. Until now, devices removed are Huawei P20 Pro, Nova 3, and Honor Play.
It is an INCREDIBLY big deal that @HuaweiMobile has had some of its devices delisted from a respected and trusted benchmark group like UL/Futuremark with 3DMark. The P20 Pro, Nova 3, and Honor Play results are removed.
— Ryan Shrout (@ryanshrout) September 6, 2018
You should never completely trust benchmarks. Never. We are not telling this to you because benchmarking platforms are not reliable, and neither because benchmarks are not the only factor to consider in order to evaluate the performance of a smartphone. You should not trust benchmarks because faking them has become a common practice, especially among Chinese smartphones manufacturers. Or at least this is what the president of Huawei’s software division, Mr. Wang Chenglu thinks. Yes, after the accusations on OnePlus and Meizu last year, even Huawei and Honor phones have been caught cheating on benchmarks to achieve higher results.
The sad discovery has been made by the guys from Anandtech, who have detected weird results on benchmarks after testing Huawei and Honor flagships, so they decided to dig deeper. An initial test has highlighted that Huawei P20 and P20 Pro obtained an inferior score on graphics benchmarks compared with the predecessor Huawei Mate 10, even though they had the same SoC (the octa-core Kirin 970) and the GPU with the same frequency. After another test, they have noticed a similar result with the new Honor Play. In a few words, these flagships are able to detect when the user is running a benchmark app and they increase their performance temporarily to obtain higher scores.
The Chinese company also used to highlight these scores to promote the devices. Anandtech revealed that the handsets are able to achieve up to 50% better results on benchmarks thanks to the trick. But while the smartphone cheats, it heats much more and energy consumption gets way higher. Chenglu has implicitly admitted the guilt at IFA 2018 and promised more transparency in the future. According to him and as we said above, faking benchmarks has become a common practice in China (he also mentioned a known Chinese manufacturer faking benchmarks) and Huawei had to adapt.
(source)
