The future of self-checkout means no check out at all
How e-commerce giant Amazon is revolutionizing the shopping experience with image sensing technology
Amazon made a splash recently when it unveiled its new checkout-free grocery store Amazon Go, based out of Seattle, Washington. The technology enables customers to shop around, pick up items and walk out without having to wait in a line up or deal with finicky credit cards. Freedom at your fingertips – all that’s required is the Amazon Go app, or so it seems.
The automated consumer
More and more, automation is seeping into everyday life. From cars to manufacturing, the human touch is being replaced with control systems that can often perform faster and more accurately.
In the ’90s, self-checkout systems were at the forefront of automated technology. Consumers wanted more freedom and flexibility during their shopping experience.
In 1992, Dr. Howard Schneider, a then emergency room doctor, installed the first-ever self-checkout system at the Price Chopper Supermarket in New York. Soon thereafter, NCR – a developer of consumer transaction technologies – prototyped their first set of self-checkout machines. By 2003, self-serve checkouts had become widespread across North America. Now, there are more than 200,000 of these devices around the world and it’s expected that number will reach 325,000 by 2021.
The problem: the technology itself hasn’t evolved much since it was first introduced.
Self-checkout hiccups
Everyone knows the frustrations that come with the self-checkout experience: you can’t find the bar code, the machine doesn’t recognize your item, the scale isn’t working, you didn’t identify the correct amount of bags you need. Now, you have to wait 15 minutes for employee assistance.
The University of Leicester has found that due to these complications, self-service checkouts turn “normally-honest” shoppers into criminals. According to their study, theft is not only easy, but it’s the simplest solution to avoid frustrating technology.
On the flip side, a survey out of retail-technology company Tensator revealed that one in three British shoppers have simply walked out of a store without purchasing the goods they intended to buy because of a poor self-service checkout experience.
And while you may think the brains behind these kiosks are to blame, the reality is, it’s incredibly challenging to sync each component of the machine to ensure it runs smoothly.
Dusty Lutz, vice-president of NCR’s store transformation solutions group, has been part of the self-checkout conversation for quite some time. In fact, in an interview with National Public Radio last October, he recalls being in the room where the all-too-familiar phrase ‘unexpected item in bagging area’ was coined.
In the same interview, he reinforced the complexity of self-checkout machines. Finding the right balance between preventing and identifying theft while also creating an enjoyable shopping experience involves many moving parts. From cameras to scales to alerts – self checkout machines are complicated beasts.
Improving this technology would no doubt benefit retailers and customers alike.
A checkout-free future
For these reasons and more, Amazon Go’s checkout-free system – while still in beta phase – is widely welcomed. If implemented properly, consumers will be able to shop seamlessly without running into design errors and service mistakes and, in turn, retailers can save on operational costs.
According to Amazon, the independent shopping experience is made possible by the same technology used in self-driving cars: computer vision, sensor fusion, and deep learning – and referenced by Amazon as “Just Walk Out Technology.”
The company filed a patent in 2014, published in 2015, that sheds light on the process. While it’s likely that changes have been made to the technology, from the introductory video, it seems it’s along the same lines as when it was first laid out three years ago.
‘Just Walk Out Technology’ relies heavily on cameras and voice recognition. When a customer enters the store, they are tagged and tracked as they walk around the space.
The system can identify products taken and returned from shelves and keeps track of the items in a user’s ‘virtual cart,’ updating when changes are made. As they leave the store payment is taken from their Amazon Go account.
As noted in the patent, “In some implementations, the image capture devices may be positioned overhead, such as on the ceiling, to capture images of users and/or locations within the materials handling facility.” It goes on to say “One or more cameras may be positioned within the inventory locations to capture images of items stored in the inventory locations and/or images of objects (e.g., a user’s hand, items) move into and/or out of the inventory locations.”
Additionally, Amazon Go relies on pressure sensors, infrared sensors, a scale, load cells, a volume displacement sensor, a light curtain, among other technologies – terms that likely don’t have much meaning to the average consumer. To help decode this, Amazon described a few scenarios where these systems will be used.
“A pressure sensor and/or a scale may be used to detect when an item is added and/or picked from inventory locations. Likewise, an infrared sensor may be used to distinguish between a user’s hand and inventory items. In another example, load cells may be positioned on a surface of the transition area to detect when a user is passing through the transition area.”
Like self-driving cars, the technology becomes more sophisticated and less prone to error as more information is collected – known in the tech world as Deep Learning. Eventually, the computer system behind Amazon Go will be able to understand their customers; from what they look like to what they like to eat.
This isn’t the first time a big corporation has envisioned a checkout-free future. In 2006, IBM released an ad about a store that’s based on the same premise as Amazon Go but with different technology.
The thinking has been around for quite some time, it’s the implementation that’s new.
Dealing with sophisticated tech
Undoubtedly, there will be hurdles along the way – hurdles that are likely to surface when the store opens its doors to the public. And given the wide-array of technologies used in Amazon Go, a seamless implementation into the “real world” seems next to impossible.
Some tech-based writers in the industry are already identifying some potential red flags. In an article published by digital intelligence platform Purple, Holly Wells questions whether Amazon’s artificial intelligence will become confused when two users are shopping in close proximity, “what if you’re charged for someone else’s item?” she said.
Another blogger notes that the nature of grocery shopping is tricky in and of itself. Shoppers tend to return items to the wrong shelf, perhaps in the wrong aisle, which might add a layer of complexity to the computing system.
Their predictions weren’t far off.
According to a recent CNET report, Amazon is delaying the launch of its smart grocery store because of technical difficulties. Supposedly, the company is having trouble tracking more than 20 customers at once and keeping tabs on the merchandise throughout the store.
Just like the very first self-checkout system, you can see that perfecting this technology is more difficult than it appears on the surface.
The use of cameras and voice recognition also raises questions around privacy. A more independent, flexible shopping experience comes at the cost of being tracked online. It’s uncertain whether users will be comfortable shopping in a space that’s calculating and analysing their every move.
Looking ahead
As more details are revealed and the first round of shoppers make their way through the Seattle store, we’ll soon know whether Amazon has successfully re-envisioned the checkout experience.
Whatever the outcome, it seems checkout-free shopping is here to stay. Consumers will always be looking for a more convenient and efficient way to navigate stores and retailers will always have an interest in reducing costs. Regardless of which side of the debate you favour, you can be certain this technology is going to continue to develop at a rapid rate- whether it’s Amazon or a new upstart we haven’t even heard of.
How mobile sensors + cloud processing are powering new consumer apps 
Image Sensor Basics 