National Academies Press: OpenBook

Airport Biometrics: A Primer (2021)

Chapter: Appendix A - Case Study: Amazon Go Cashierless Retail Experience

« Previous: Glossary
Page 135
Suggested Citation:"Appendix A - Case Study: Amazon Go Cashierless Retail Experience." National Academies of Sciences, Engineering, and Medicine. 2021. Airport Biometrics: A Primer. Washington, DC: The National Academies Press. doi: 10.17226/26180.
×
Page 135
Page 136
Suggested Citation:"Appendix A - Case Study: Amazon Go Cashierless Retail Experience." National Academies of Sciences, Engineering, and Medicine. 2021. Airport Biometrics: A Primer. Washington, DC: The National Academies Press. doi: 10.17226/26180.
×
Page 136
Page 137
Suggested Citation:"Appendix A - Case Study: Amazon Go Cashierless Retail Experience." National Academies of Sciences, Engineering, and Medicine. 2021. Airport Biometrics: A Primer. Washington, DC: The National Academies Press. doi: 10.17226/26180.
×
Page 137
Page 138
Suggested Citation:"Appendix A - Case Study: Amazon Go Cashierless Retail Experience." National Academies of Sciences, Engineering, and Medicine. 2021. Airport Biometrics: A Primer. Washington, DC: The National Academies Press. doi: 10.17226/26180.
×
Page 138
Page 139
Suggested Citation:"Appendix A - Case Study: Amazon Go Cashierless Retail Experience." National Academies of Sciences, Engineering, and Medicine. 2021. Airport Biometrics: A Primer. Washington, DC: The National Academies Press. doi: 10.17226/26180.
×
Page 139
Page 140
Suggested Citation:"Appendix A - Case Study: Amazon Go Cashierless Retail Experience." National Academies of Sciences, Engineering, and Medicine. 2021. Airport Biometrics: A Primer. Washington, DC: The National Academies Press. doi: 10.17226/26180.
×
Page 140
Page 141
Suggested Citation:"Appendix A - Case Study: Amazon Go Cashierless Retail Experience." National Academies of Sciences, Engineering, and Medicine. 2021. Airport Biometrics: A Primer. Washington, DC: The National Academies Press. doi: 10.17226/26180.
×
Page 141
Page 142
Suggested Citation:"Appendix A - Case Study: Amazon Go Cashierless Retail Experience." National Academies of Sciences, Engineering, and Medicine. 2021. Airport Biometrics: A Primer. Washington, DC: The National Academies Press. doi: 10.17226/26180.
×
Page 142
Page 143
Suggested Citation:"Appendix A - Case Study: Amazon Go Cashierless Retail Experience." National Academies of Sciences, Engineering, and Medicine. 2021. Airport Biometrics: A Primer. Washington, DC: The National Academies Press. doi: 10.17226/26180.
×
Page 143
Page 144
Suggested Citation:"Appendix A - Case Study: Amazon Go Cashierless Retail Experience." National Academies of Sciences, Engineering, and Medicine. 2021. Airport Biometrics: A Primer. Washington, DC: The National Academies Press. doi: 10.17226/26180.
×
Page 144

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

135   Summary Initiated by Amazon, the Amazon Go retail experience (see Figure A-1 and Figure A-2) uses a combination of cameras, sensors, computer vision techniques, machine learning, and artificial intelligence to create a cashierless retail experience; the first airport location will be at Newark Liberty Inter national Airport (EWR) (Amazon Go 2020). This technology, called “Just Walk Out,” requires customers to identify themselves when they enter the store (Amazon 2020). The customer’s identity is verified either by scanning the Amazon Go mobile app (in the case of an Amazon Go store) or their credit card (for other retailers equipped with the Just Walk Out technology). A biometric variant also exists, where instead of a QR code or credit card, customers enroll at the storefront enrollment kiosk by scanning their 3D hand palm biometric and linking that to their account. The Just Walk Out technology leverages computer vision and machine learning to distinguish between customers and the items picked out and added to their virtual cart. The system does not rely on facial recognition but relies on movement tracking. Each customer is associated with a name, an account, and a consumer profile that reflects all interactions with items on the shelves (which items are stared at, picked up, or bought). Future use of the customer’s data is still undefined. However, the initial patent application included some examples where the customer’s purchase history could be used to confirm which items were picked by the user. The main benefits are significant time savings for customers and financial savings for retailers (reduced to no cashiering cost) (Bishop 2020b). The system architecture relies on Amazon’s A P P E N D I X A Case Study: Amazon Go Cashierless Retail Experience Source: Adapted from Amazon Go 2020. Figure A-1. Amazon Go shopping experience.

136 Airport Biometrics: A Primer What? ● Cashierless retail technology ● Project implemented by Amazon Where? ● 26 Amazon Go stores across the United States (e.g., Chicago, New York, Seattle, San Francisco) ● Deployment at a few airports across the United States in stores of retail chain OTG (On-the-Go), including at EWR (OTG Management 2020) Customer process steps ● To access Amazon Go stores: – Create an Amazon account – Download Amazon Go app – The Amazon Go mobile app interface will generate a QR code, or the user may add a hand palm scan to the account at an enrollment kiosk. – Customer enters the store by scanning the QR or hand palm (or credit card for other retail stores with Just Walk Out technology) – Just Walk Out technology tracks the customer and the interaction with store items. – Virtual cart validation – Exit Who? ● Amazon patented the Just Walk Out technology. It provides the hardware (camera and sensors) along with the software. Why? ● Amazon’s goal is to create a seamless shopping experience where customers do not have to wait in line. How? ● The concept relies on the combined use of cameras, sensor fusion, computer vision, and deep-learning algorithms to track customers during their time in the store, noting each item picked up, put back, or added to their virtual carts. For the biometric hand palm variant, 3D hand palm scanners and a palm recognition software are used. Enrollment/digital identity creation and verification ● An Amazon account can be made online, and enrollment is done in the storefront at the enrollment kiosk. The online Amazon account can be linked to the hand palm biometric by scanning one's hand at the kiosk. Verification of identity how? ● Customers are identified by either a QR code generated by the Amazon Go app in Amazon Go stores or their credit card in third-party retail stores where the Just Walk Out environment is available. For? ● Amazon account holders Table A-1. Key facts for the Amazon Go case study. Just Walk Out technology and includes the camera and sensor hardware as well as the software system. Table A-1 contains a summary of the Amazon Go case study. How Does It Work? Before the Customer Journey • Amazon Go relies on sensor fusion (analyzing and aggregating data from multiple sensors, including weight and movement sensors), advanced data hosting services through Amazon Web Services, and advanced computer vision–based machine learning (Amazon 2015). • The hardware includes electronic shelves, cameras, fixtures, and a facility management system. • The inventory management involves data storing and item identification. • In the case of Amazon Go stores, customers create an Amazon account and download the Amazon Go app. • In the case of Just Walk Out–enabled stores, customers access the store with their credit card.

Case Study: Amazon Go Cashierless Retail Experience 137   The Customer Journey The customer journey can be described by the following: • Each shopper (and that shopper’s party) enters and is identified with a QR code generated by the Amazon Go app, biometric hand palm, or a credit card in the case of stores equipped with the Just Walk Out technology. • The technology tracks the customer’s movements and interactions with the different store items. As customers remove items from the shelves, those items are added to their virtual cart (Kumar et al. 2013). • The customer receives a receipt and is charged when exiting the store. Retention and Storage Account data are saved on Amazon’s own servers. • Cameras track throughout the store in real time using the video feed streaming from a ubiquitous network of cameras (Bacco and Hiatt 2010). • Amazon Go stores retain data on the interactions of individual customers with the items on the shelves. These data are used to further train the artificial intelligence running the architecture. • Account data are retained and stored at Amazon until the passenger opts for the deletion of the account. • Customers can elect to delete their biometric data. System Architecture Flow Diagram The flow diagram for this case study can be found in the Amazon Go Cashierless Retail Experience case study of Chapter 2. Source: Amazon 2020. Figure A-2. Just Walk Out enables seamless store experiences for other retailers as well.

138 Airport Biometrics: A Primer System Specifications The system relies on an artificial intelligence (see Figure A-3) to track the customer in the store. The artificial intelligence architecture relies on Amazon Web Services for streaming services as well as advanced computer vision–based machine learning. Computer Vision–Based Machine Learning To associate the right items with the right customers, the technology relies on aggregating data from different sensors linked with the location information of the sensors. The artificial intelligence driving the architecture tracks the customers in the store by aggregating data from different sensors through sensor fusion and solving different identification and linking problems to associate customers, their location, and their interactions with items in the store. Person Detection The Just Walk Out technology does not use facial-recognition technology but relies on red- green-blue cameras equipped with depth- and distance-sensing capabilities. Each customer is associated with a general profile and an anonymized 3D point cloud. During the customer’s time in the store, a deep-learning algorithm predicts the customer’s location and associates that location with actions and interactions with store items (see Figure A-4). System Architecture, Pre-Existing Systems, and Databases The following explain the buildup of the system and its processes: • Customer information and biometrics are collected at the entry gate. • The identification of distinctive features on the hand(s) of the customer relies on a propri- etary algorithm, with distinctive features of the hand specific to this proprietary technology architecture (see Figures A-5 through A-7). • Hand biometrics data that are collected are either stored, if the customer has an Amazon account, or are deleted once the customer exits the store. • The customer holds his or her palm over the device to opt in. Source: Amazon 2019. Figure A-3. Amazon artificial intelligence for the Amazon Go retail experience.

Case Study: Amazon Go Cashierless Retail Experience 139   Stakeholders and Responsibilities Stakeholders The main stakeholders of Amazon Go concept are Amazon and the third-party retailers who have purchased Just Walk Out technology services. Responsibilities Amazon owns the Just Walk Out concept, which builds on a series of patents submitted by Amazon Technologies since 2013 (Puerini et al. 2014). Case Study Review Benefits The Just Walk Out technology presents benefits both for the customer and the retailer. Benefits to customers include: • Improved customer experience over time with deep learning allowing for faster tracking and more accurate identification of interaction with store items, • Limited contact and interaction with store employees, • Reduced time spent in the store, and • No use of facial recognition. Benefits for the retailers providing the Just Walk Out–enabled experience include: • Adaptability of the Just Walk Out technology to different store layouts, • Financial savings due to reduced staff cost related to cashiering or inventory, • No use of facial recognition, • Improved customer experience, and • Detailed consumer profile data. Source: Amazon 2019. Figure A-4. Logic structure of Amazon Go and technology components.

140 Airport Biometrics: A Primer Source: Kumar et al. 2018. Note: Numbers in figure are part of the patent application and are not relevant to this discussion. Figure A-5. Hand biometrics identification process.

Case Study: Amazon Go Cashierless Retail Experience 141   Source: Kumar et al. 2018. Note: Numbers in figure are part of the patent application and are not relevant to this discussion. Figure A-6. Hand biometrics characteristics identification.

142 Airport Biometrics: A Primer Source: Kumar et al. 2018. Note: Numbers in figure are part of the patent application and are not relevant to this discussion. Figure A-7. Person detection based on hand biometrics identity.

Case Study: Amazon Go Cashierless Retail Experience 143   Responses From Customers Crowd-sourced reviews for individual stores rate on average more than 4 stars out of 5. A stated choice survey conducted by the Shorr group in 2018 found that “84% of respondents to a survey said that they see Amazon Go as a ‘type of grocery shopping experience’ they’d enjoy more than traditional grocery shopping” and “over 25% of respondents said that they would pay more for grocery products if it meant they didn’t have to wait in line at checkout” (Shorr 2018). System Performance and Specifications Review The Just Walk Out technology requires an approximately $1 million investment in hard- ware. Customers spending less time in the store allow for higher customer throughput per hour compared to a traditional store (Cheng 2019). The system is designed for 99% accuracy, including during the busiest periods. On average, Amazon Go stores process 550 customers per day and as many as 90 people at any given time. Fall-Back Options • Despite the absence of cashiers, a few store staff members are present to assist customers with any technical issues, answer questions, and process cash transactions. • If customers are incorrectly charged for an item, they can contest the charge and be reimbursed. Concerns • Privacy and use of collected data: There is a lack of transparency on how the collected data factor into the overall Amazon personalized marketing strategy. The architecture relies on deep learning, and data mining of the customer’s activity in the store would improve the experience and accuracy over time. • Lack of transparency around the use of data slowed the adoption of the technology in Europe because of confrontation with GDPR regarding consent to the processing of personal data. In the context of the regulation, every consent request must state the precise purpose for which the data will be processed (Walters 2020). Lessons Learned The Just Walk Out technology initially relied on motion tracking and an anonymized cloud of data to identify individual customers. However, in December 2019, Amazon submitted a patent for a non-contact biometric identification system to identify customers by their hands (Kumar et al. 2018). Introducing hand biometrics as one of the identifying features of each customer is expected to simplify the person-detection component of the Just Walk Out architecture. Findings and Trends Findings The Just Walk Out technology is a potential template for future models of touchless and cashierless retail at airports and other shopping locations. The use of biometrics for identifica- tion, camera tracking devices, image analysis software, and a program that automates product billing has proven to create a new user-friendly experience that is more efficient and is touch free and seamless. Amazon now has several versions of this retail technology in operation, and future revisions of the concept will only improve accessibility. Enrollment challenges may be resolved with biometric palm geometry recognition, which is also being trailed, or with a simple

144 Airport Biometrics: A Primer two-step authentication process. Another option is the use of not just one biometric, but two or more, thereby providing options for customers to choose from. The main benefits behind using this type of software are significant time savings for customers and financial savings for retailers through lower operational expenditures on employees (e.g., labor wages, benefits). Some enrollment challenges may be resolved with simple two-step palm enrollment, which is interesting because it may seem less invasive when compared to a picture of one’s face. Furthermore, this type of software presents a potential alternative to current and future models for touchless retail for airports and airlines. Future Situation and Broader Implementation Amazon submitted a patent application for a touchless hand scanning system named “Amazon One” and started implementing it in Amazon Go stores in October 2020. Customers scan their hands at the entry gate to enter the store. Trends Identified A trend identified in the Amazon case study is that the use of hand biometrics is aimed to facilitate a more efficient, user-friendly (retail) experience. In the cashierless system, time is saved, shopping requires less fumbling of personal items during payment, and simplicity for the user can bring satisfaction. In this specific case, it is not only the biometric technologies enabling this, but also the camera tracking system and software, which allow for the automatic charging of customers to their accounts. This is a trend that will likely expand to other sectors outside of retail and airports. Similarly, the retail experience removes the need for contact points and human interactions, which has the added benefit of reducing the risk of transmissible diseases, which is especially beneficial in the COVID-19 era. The camera system and the 3D hand geometry scanners allow for tracking and identification without customers needing to touch any surfaces.

Next: Appendix B - Case Study: Denver Daon Biometric Partnership »
Airport Biometrics: A Primer Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Biometrics is one of the most powerful, but misunderstood technologies used at airports today. The ability to increase the speed of individual processes, as well as offer a touch-free experience throughout an entire journey is a revolution that is decades in the making.

The TRB Airport Cooperative Research Program's ACRP Research Report 233: Airport Biometrics: A Primer is designed to help aviation stakeholders, especially airport operators, to understand the range of issues and choices available when considering, and deciding on, a scalable and effective set of solutions using biometrics. These solutions may serve as a platform to accommodate growth as well as addressing the near-term focus regarding safe operations during the COVID-19 pandemic.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!