Just last week, DxOMark began rating the selfie camera(s) in phones. Now popular benchmarking tool AnTuTu has announced an AI benchmarking tool.
With AI performance being a much-advertised feature of processors and phones these days, we can say the tool has arrived just in time. AnTuTu says the tool is still in beta.
In an official press release, AnTuTu revealed there is still no unified standard for AI as each chipset manufacturer sees it differently. Qualcomm uses its DSP to handle AI tasks while Huawei, Samsung, and MediaTek have a dedicated AI unit called NPU (APU for MediaTek). All the chip manufacturers also have different SDKs too.- NeuroPilot for MediaTek, SNPE for Qualcomm, and TensorRT for NVIDIA. Samsung hasn’t announced its own yet.
Nevertheless, they have been able to develop a unified standard for testing thanks to cooperation from processor manufacturers. The test is said to be divided into two categories – image classification and object recognition.
Image Classification is based on Inception v3 neural network and the test includes 200 pictures. Object Recognition is based on mobilenet SSD neural network and its test is a 600-frame video.
AnTuTu advises that the test should be done using Wi-Fi as the data packet is about 160MB in size even though the app itself is just 33MB. Downloading the data packet on a device you intend to test is a one-time thing even if you plan to run the test multiple times. However, there will be a data packet verification every time.
Devices are ranked based on a combination of speed and accuracy. Hence, a device that is fast but not accurate will rank lower. AnTuTu says there are other conditions that may make a device score lower.
READ MORE: Lenovo Z5 Pro GT records highest benchmarking score on AnTuTu
For example, a device running an older Android version will score less than one running a more recent one. This is because Google optimizes AI performance with each new version of Android released. Also, the fact that Samsung hasn’t released its own SDK yet will result in lower scores for its devices. There will also not be significant differences between two processors that use the same “AI processor”. An example is the Snapdragon 710 and Snapdragon 845 which both use the Hexagon 685 DSP.
