Watch all the Transform 2020 sessions on-demand here.
Amazon shared more details today about Amazon Go, the company’s brand for its cashierless stores, including the use of synthetic data to intentionally introduce errors to its computer vision system. Challenges that had to be addressed before opening stores to kill queues include the need to make vision systems that account for sunlight streaming into a store, little time for latency delays, and small amounts of data for certain tasks.
Synthetic data is being used in a number of ways to power few-shot learning, improve AI systems that control robots, train AI agents to walk, or beat humans in games of Quake III.
“As our application improved in accuracy — and we have a very highly accurate application today — we had this interesting problem that there were very few negative examples, or errors, which we could use to train our machine learning models,” Dilip Kumar, VP of Amazon Go, said. “So we created synthetic datasets for one of our challenging conditions, which allowed us to be able to boost the diversity of the data that we needed. But at the same time, we have to be careful that we weren’t introducing artifacts that were only visible in the synthetic data sets, [and] that the data translates well to real-world situations — a tricky balance.”
Kumar spoke onstage today at the Amazon re:Mars conference, which stands for Machine Learning, Automation, Robotics, and Space.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
Also announced today: Alexa Conversations for seamless interactions with Alexa skills using deep learning, new warehouse robots, and plans to begin drone deliveries in the coming months.
Synthetic data is also being used to improve Amazon drones and warehouse robots that made their debut today, Amazon worldwide consumer CEO Jeff Wilke told VentureBeat in a press conference. Echoing Kumar, Wilke said this requires engineers to perform audits to ensure desired outcomes take place.
“Nearly everything that we talked about that [during the keynote address] is using large datasets to train models and create inference gets better if you have simulated data. The problem, of course, is that you can end up with bias in simulated data and blind spots and edge cases and so on,” Wilke said.
Above: Amazon VP Dilip Kumar shares a map of a Go store onstage at re:Mars at the Aria Hotel in Las Vegas on June 5, 2019.
In addition to the use of synthetic data and trials with human customers, Amazon engineers made computer vision algorithms that utilize both geometry and deep learning to ensure Go stores accurately associate transactions with the right customer. To meet the need for crunching data from videos in real time, Amazon also created Kinesis, a service that has since been incorporated into AWS RoboMaker.
“Because people are moving around continuously, actions are happening continuously, and when people get very close to each other, there’s simultaneous actions that are happening where multiple people are taking items in very close proximity to each, [and] other events that are very close in space and time,” Kumar said.
Amazon first introduced the idea of a cashierless store to the world in 2016 with the opening of a trial location near its company headquarters in Seattle. Amazon Go now has 12 locations, in Seattle, Chicago, San Francisco, and New York.
To support people without bank accounts or debit cards — or those whose phone batteries are dead and cannot access the Amazon Go app — Amazon announced plans in April to accept cash payments like traditional stores.
