Facebook is launching what it’s calling a “universal product recognition model” that uses artificial intelligence to identify consumer goods, from furniture to fast fashion to fast cars.
It’s the first step toward a future where the products in every image on its site can be identified and potentially shopped for. “We want to make anything and everything on the platform shoppable, whenever the experience feels right,” Manohar Paluri, head of Applied Computer Vision at Facebook, told The Verge. “It’s a grand vision.”
Product recognition is the first in a slew of AI-powered updates coming to its e-commerce platforms in the near future, says the company. Eventually, these will combine AI, augmented reality, and even digital assistants to create what it calls a “social-first” shopping experience. In addition, it also today launched a feature called Shops, that lets small businesses set up free storefronts on Facebook and Instagram.
:no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/19986894/Screen_Shot_2020_05_19_at_6.14.46_PM.png)
Fashion will be a key part of this, with the company suggesting that a future Facebook “AI fashion stylist” could offer users personalized shopping recommendations based on their wardrobe and daily suggestions for outfits tailored to the weather and their schedule.
“This is something I’ve wanted to build since I watched the movie Clueless,” Tamara Berg, a Facebook research scientist, told The Verge. “They really imagined everything back in 1995, but the technology I think is now finally ready to make it come to life.”
But as the Clueless reference suggests, these features aren’t exactly new ideas. Even outside of Hollywood, they’ve been tried and tested for years, often with mixed results.
Amazon already built its own AI-powered fashion assistant with the Echo Look, now little heard-of. And using machine vision to identify and shop for products has been a reality since at least the Amazon Fire Phone. Meanwhile, online shopping platforms like eBay already use AI to speed up the process of listing items for sale, and Amazon is one of a number of firms that’s launched its own “Shazam for clothes” using machine learning.
:no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/19986830/facebook_ai_assistant.png)
Facebook says what makes its tools different are their scope and accuracy. The company’s new product recognition tool, GrokNet, can identify tens of thousands of different attributes in an image. These range from specific brands to things like color and size.
GrokNet has already been deployed on Facebook Marketplace, where it helps users quickly list items for sale by identifying what’s in them and generating short descriptions. You might upload a photo of your couch, for example, and Marketplace will suggest listing it as “black, leather, sectional sofa.”
The company is also testing a version of this tool that’s built for businesses. When they upload photos to their page containing their own products, the AI system can automatically tag them and link to shopping pages.
In building these tools, Facebook is helped by its access to users’ photos on Marketplace. GrokNet is trained on a colossal database on the order of magnitude of around 100 million images, with the majority taken from Marketplace. Facebook says this data is vital in creating a machine vision system that can identify products in challenging lighting and from dodgy angles — a part of the online shopping experience that isn’t going away.
It’s unclear, though, exactly how accurate GrokNet is. The company says it can identify 90 percent of images on Marketplace in the Home and Garden category, but it didn’t give similar statistics for other types of product categories.
As is often the case with tools like this, the difference between the advertised features and actual user experience can be huge, and we’ll have to wait and see what reaction GrokNet gets from Facebook’s users.
