Facebook is banking on the smartphone camera being the most influential and ubiquitous augmented reality platform in the future, and it’s spent a healthy chunk of its F8 developer conference showing the world what this will look like. While virtual game boards and underwater effects are neat, the company is also developing AR features that are more subtle and way more useful.
One involves using a mix of the same type of computer vision and artificial intelligence-powered algorithms that place virtual images in a scene to do standard photo effects. So instead of placing a giant whale next to your cereal bowl, the camera emulates a low-aperture photo with a blurred-out background. Onstage at F8, Facebook’s Joaquin Candela, the head of its applied machine learning team, showed how its software is capable of “dynamically estimating what is foreground and what is background.” Once it differentiates between the two, it can blur on out to produce the desirable bokeh effect you get with professional DSLR camera with a low-aperture lens.
This is the same type of technology Apple introduced with its iPhone 7 Plus’ dual-camera setup, yet Facebook accomplishes this without needing a second lens for depth information. Instead, it uses its more powerful computer vision and depth sensing capabilities to do this, and let it work in real time. In the demo, Candela explained how you can swing the viewfinder of Facebook’s AI-powered camera around and still retain the blurred out background. (Apple’s iPhone 7 Plus camera often requires you to stay still while it detects the subject.)
Of course, this isn’t the first smartphone camera effect to simulate high-end camera features without relying on hardware. Numerous apps, typically with “blur” in the name, do the same, and even Instagram’s more basic editing features let you do a post-processing tilt-shift effect that lets you blur out distinct portions of a photo, to produce a miniature effect or draw focus to one part of a photo.
Yet none of these seem as capable as Facebook’s, likely because none are backed up by the same type of AI and augmented reality software on the backend. So while Facebook isn’t the first to give users this neat feature, the ubiquity of its mobile software and its insistence on pushing camera features in every one of its apps will likely mean Facebook’s toolkit becomes the most popular.