testsetset
When Google’s Pixel 3 and Pixel 3 XL debuted two months ago, our Pixel 3 review concluded that AI is a defining feature of these phones. That said, my colleague Kyle Wiggers followed up with an op-ed noting that Google’s camera decisions on the Pixel 3 suggested AI can’t do everything. The camera rankings DxOMark released today add fuel to this fire.
Let’s get the methodology out of the way: “For scoring and analysis in our smartphone camera reviews, DxOMark engineers capture and evaluate over 1,500 test images and more than 2 hours of video both in controlled lab environments and in natural indoor and outdoor scenes, using the camera’s default settings.” In other words, while this is clearly a thorough camera test for both photo and video, keep in mind it is just one test.
DxOMark only focuses on the rear camera. Google went with dual front-facing cameras this year, but it kept a single one on the back. The argument was that AI would handle it all, even if it apparently couldn’t on the front. Here’s Kyle, grappling with this in October:
I’ve been trying to reconcile the inconsistency since the Pixel 3 was announced last week. Is the addition of a second front camera a tacit admission that AI can’t do everything? That a physical camera is superior? That a hybrid approach might be best?
We now know AI couldn’t keep up with phone camera advances in 2018. Google admitted as much on the front, but couldn’t get itself to do the same on the rear. And it shows.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
The Pixel is no longer king
The original Pixel dominated the DxOMark charts in 2016:
As did the Pixel 2 in 2017:
This year’s Pixel topped the charts only if you’re looking at single rear-facing cameras. Otherwise, it’s No. 8:
From DxOMark’s conclusion:
With the Pixel 3, Google continues to make impressive gains in image processing using computational imaging. The result is a very versatile camera that people can confidently use in most situations without any manual settings or tweaks. In particular, the Pixel 3’s Zoom and Bokeh (Portrait mode) scores have improved dramatically over the previous generation. Overall, it is tied for the top of our single-camera phones with the Apple iPhone XR and beats it, along with nearly every other phone in our database, for Video. The Pixel 3’s scores are behind those of only a handful of multi-camera flagship phones.
For all its advances in computational imaging, and improvement in Bokeh and Zoom scores, the Pixel 3 is still behind the top multi-camera phones in those areas. So for buyers looking for the ultimate in Portrait photography, and who also don’t mind the larger size of one of the multi-camera flagship models, it may be worth considering one of them as an alternative.
Whatever DxOMark says, Google’s AI camera gains continue to blow the competition away. For proof, look no further than Night Sight.
It’s quite possible the Pixel 4 will retake the DxOMark crown, even if Google keeps a single rear camera. The problem isn’t that Google is using AI, it’s that the competition is too. They’re adding extra cameras and AI features.
If, however, the Pixel 4 has multiple rear cameras, it will definitely dominate again.
ProBeat is a column in which Emil rants about whatever crosses him that week.
