How Clarifai Envisions Commerce — Literally

Industries around the world have caught AI fever. Developers around the world have made more ways than ever for the technology to automate, optimize and enable different services.

Facebook, Alphabet, IBM, Microsoft, Amazon and other major companies are all working on AI projects, along with numerous tech startups. One such upstart, which leverages artificial intelligence and image recognition in part to enable commerce, is Clarifai.

Founded in 2013, Clarifai utilizes neural networks and provides customers with an image and video recognition API. The company currently has 39 employees and closed a $30 million Series B funding round in the fall.

Matthew Zeiler, founder and CEO of Clarifai, told PYMNTS’ Karen Webster in a recent interview that he wants Clarifai to remain an independent AI company.

“Since starting Clarifai,” Zeiler said, “we’ve seen many startups in the space that have since been acquired.” He noted that, while Clarifai celebrates acquisitions and the successes of other startups, his company is aiming at bigger plans.

“There has to be an independent company,” Zeiler said. “Because now our competitors are Google, Microsoft, IBM — tech giants — but none of them are independent of their customers. They all have divisions and products that compete with them.”

Zeiler said he wants Clarifai to be different, to focus on working with customers rather than on creating products to compete with them. “We work with them to help solve problems,” he said.

Clarifai works with clients of all sizes, from Fortune 500 enterprises down to individual developers, Zeiler said. Since starting Clarifai, he noted that the company has gone in a few different directions in terms of what the image recognition technology can offer.

Clarifai offers customers an image and video recognition API that users can teach to automatically tag, organize and search visual content using machine learning. “If you can imagine any sort of visual state of the world,” Zeiler said, “you can teach our platform to recognize it.”

“We started with one model that tries to recognize as many different objects in the world as possible. Today, it recognizes over 11,000 different things,” Zeiler said, noting that the model also recognizes the objects in 23 different languages, including English, Mandarin, Japanese, Russian and other major world languages.

For video, Clarifai’s API doesn’t just send back one set of tags. “We give you a timed series of information throughout the video, showing what we see at any given time,” said Zeiler. “And it happens much faster than a human can do it.”

Beyond the comprehensive model, Clarifai offers niche models for industries that require more specific classifications. “We built what we call domain models,” Zeiler said. “We have them for travel, weddings, food, and fashion is coming up soon.”

On retailers’ websites, Clarifai’s image recognition API can help brands categorize and upload new products more quickly.

“We can help them get new products onto their page quicker,” Zeiler said, “and make more consistent taxonomies and categorizations for their products.” With Clarifai’s AI capabilities, the technology learns how individual retailers want to tag their content, eventually allowing for automatic input.

Clarifai also has a moderation domain model for sites where users upload their own content. A specialized engine looks for evidence of nudity, illicit drugs, weapons and other content that websites may want to monitor for.

Zeiler noted that Clarifai sits on the upload buttons of many eCommerce market sites, scanning images to seek out unwanted content.

“We help you filter all of that out,” he said. “It doesn’t even have to be offensive — just unrelated to the site. For instance, they don’t want stock photos because they want to know you own the product. It helps you speed up the upload process and make a better site for your users.”

Brands can also leverage Clarifai’s image recognition API to collect consumer data and enable what Zeiler calls social understanding.

“We have a lot of customers who want to go to social media to understand how their brand or products are being talked about,” Zeiler said. “And a lot of that information is not in the hashtags, and it’s not in the messages that people send. It’s really in the images, and now, they can understand it.”

Zeiler said that the capabilities and use cases for Clarifai’s AI-based image and video recognition can sometimes be difficult to describe.

“We’ve had demos on our website from day one,” he said. “It’s something new, or it’s something that people may know about but don’t believe in because it has never worked before. We have a demo, so you can try it with your own images or images we already have. The API results come back in real time, in fractions of seconds.”

Use cases abound. Zeiler said a developer using Clarifai’s API is developing an intelligent baby monitor to recognize when a child is standing, sitting, sleeping or not in the crib. The functions of Clarifai’s API will continue to grow as the company adds more clients and grows its offerings in the coming months and years.

“We take care of it from the pixels up,” Zeiler said. “We understand it, we index it, make it efficient for search, and we even include a user interface so that you can manage your data without needing to be a developer.”