Using overhead cameras, scanners and weight sensors to track customers in-store may be the future of physical retail, but what starts as a way to collect data and sell more via seamless checkout can turn creepy for customers. In this month’s Intelligence of Things Tracker, Krishna Motukuri, CEO of Zippin, discusses how retailers can pair in-store IoT tech to offer easy, checkout-free experiences, while giving customers enough privacy to avoid the creep factor.
When consumers shop online, eCommerce merchants can gain a treasure trove of insights into what their customers might be thinking and how they can better appeal to them, based on data such as which products they viewed and which ones they ultimately put in their carts. But when customers shop in the physical world, that data can be more difficult to collect.
Some retailers are blending digital features into brick-and-mortar environments to close that knowledge gap. Retail giant Amazon shook the physical commerce space in 2016 when it announced its Amazon Go stores. These venues require shoppers to scan an app to enter, with in- store technology tracking and automatically charging their app accounts for any items they leave with, providing a faster, cashier-free shopping experience – and, many observers noted, likely supplying Amazon with useful customer behavior data.
Others have flocked to the cashierless shopping space, to either fill different niches or go head-to-head with Amazon. That includes Zippin, a San Francisco-based startup that creates software to support stores that are interested in going checkout-free and collecting in-store customer shopping data.
PYMNTS recently caught up with Zippin CEO Krishna Motukuri to learn how he designed the company’s solution, why he believes brick-and-mortar retail is changing and why tracking customers with overhead cameras isn’t actually creepy.
Digging Into Design
One of Zippin’s main goals is to ensure that checkout lines won’t turn a quick trip to the convenience store for a few items into a long ordeal that could dampen customers’ willingness for frequent shopping.
“A line is always a big deterrent to customers,” Motukuri said. “[With Zippin’s solution], we expect customers will walk into a store more often and actually pick up things and walk out. [We expect] shorter sessions [and] potentially smaller baskets for each trip, but a substantially greater number of trips.”
The company’s approach requires consumers to scan an app to enter the store, and payment instruments linked in the app are automatically charged when they leave, a la Amazon Go. Customers can currently link cards, and Zippin is considering adding mobile wallets as a payment option.
Zippin uses overhead cameras to track customers as they move through the store, and weight sensors to determine when shoppers pick up items or put them back on the shelf. There are shortcomings to each of these technologies, however. The cameras only have a bird’s-eye view, so they may not see a small item if a consumer’s hand is covering it, and the shelf sensors cannot easily detect items that are too lightweight. When working together, however, one technology compensates for the other’s blind spots.
“If an item is small and can fit in your palm, the cameras may not see it,” he said. “[Then] the system has to rely on weight sensors. In that case, the item must weigh at least 30 grams,” or about one ounce.
These technologies use commodity hardware that is intended to better fit the price ranges of small stores, compared to the more sensor- and camera-heavy deployment of operations like Amazon Go. Motukuri claims hardware and software setup costs per square foot would be “less than 10 percent of the typical buildout cost for a store.”
While leveraging users’ phones to track their locations in the store may have dropped the price further, it isn’t precise enough to pinpoint locations in small areas. Bluetooth beacons were also out of the question, Motukuri said, as they drain too much power and don’t provide enough resolution to work well in crowded stores.
Motukuri anticipates that Zippin’s solution will likely be used in small areas to keep costs low. He also expects it to appeal to those with smaller footprints, or larger retailers looking to deploy the tech in a subsection of their stores designated for selling fast-moving products. An application like that would allow that section to remain open and operational after the main store closes.
Customer Browsing Data — for Brick and Mortar
Overhead cameras provide retailers with data on where customers stop, and what items, shelves and marketing displays they see. Combined with the shelf sensors, they’re given insight into what items consumers pick up and examine, and whether they eventually put the item back or bring it home.
“For every customer that walks into the store, you would not only know who [they are], but also exactly what they did in the store, which shelves they stopped in front of and which marketing messages they read. We can estimate the pose of every customer, so we know which way they are looking,” Motukuri said.
Customer tracking can also help prevent shoplifting, he added, as the system detects when items are picked up, and only customers registered in the app and their accompanying guests can enter the store. Additionally, sensor information can help operations with their inventory management, by providing merchants and staff with shelf-level details in real time.
Buying from Big Brother?
No store will reap revenue if it creeps out its customers. To ease customer concerns about video surveillance, cameras are not aimed at customers’ faces and do not use facial recognition or other biometric measures to distinguish between shoppers. The system is instead designed to tightly track customers based on their movements from the moment they scan into the store. It updates tracking data every 30 milliseconds to avoid losing track of a customer or risk confusing two people with the same hat, for instance.
To further put minds at ease, the app also offers a “refund” button, should customers believe they were wrongfully charged for an item. After requesting a refund, customers may be asked to return the item to a store greeter, or they may even be allowed to keep it.
Motukuri has high hopes for these solutions, predicting that customers will soon start to expect, or even demand, checkout-free options.
“We expect pretty much all stores to be checkout-free,” Motukuri said. “Once they’re used to this experience, customers won’t want any kind of self-checkout, or anything where they’re required to do a lot of work.”
If these kinds of solutions do take off with customers, retailers may find the trend brings them new convenience as well: access to richer shopper insights in their brick-and-mortar stores.
We’re always on the lookout for opportunities to partner with innovators and disruptors.
Learn More