I/O 2017: Everything coming to Google Assistant

Today was hands-down the biggest day for updates and changes to Google Assistant since the intelligent assistant was announced a year ago.

Here are the coming Assistant updates that we’ve spotted thus far at I/O, from free Google Home phone calls and a standalone Google Assistant iPhone app to an App Directory and payments for Google Assistant voice apps.

Google Assistant now on 100 million devices

That number is likely to rise when Google Assistant becomes available on Android phones in Portuguese, Japanese, German, and French.

In addition to being available on Pixel smartphones and Google Home, Google Assistant came to iPhones today and to Android phones earlier this year.

Google Assistant launches App Directory

The Google Assistant App Directory will showcase apps that interact with the Google Assistant.

“Users can access the app directory with a single tap from the Google Assistant,” said product manager Brad Abrams. “It not only has categories and user ratings, it also allows users to try your app right from the directory.”

Users can also create their own shortcuts to apps.

Since the Action on Google platform opened to developers last December, roughly 250 Google Assistant actions have been created by third-party developers.

At Facebook’s F8 and Microsoft’s Build in recent weeks, both companies have taken steps to address the need to improve bot discovery.

“Improving discoverability is important to all of us, and you can expect ongoing investments and improvements in this area,” Abrams said.

Google Assistant app for iPhone

A standalone Google Assistant app will be rolled out for the iPhone, Google announced today. Until now, Google Assistant has been available through the Allo chat app.

In February, Google Assistant was made available for Android devices with Marshmallow or Nougat installed. Altogether, Google Assistant can now be found in 100 million devices.

Google Home phone calls and push notifications

Google Home devices will be getting the ability to make free phone calls within the United States.

In the past week, Microsoft and Amazon had declared intentions to bring calls to their smart speaker devices for Alexa and Cortana.

“You can call any landline or mobile phone in the U.S. or Canada completely free, and it’s all done in a hands-free way,” said Google Home VP Rishi Chandra.

Few details were offered about push notifications.

Add events to calendar and type messages to Google Assistant

Google Home can now add events to your calendar and type messages directly to Google Assistant instead of relying only on talk.

Both of these features were requested by users almost immediately after Google Assistant on Google Home became available last fall.

Visual responses from Google Assistant on TV with Chromecast

The ability for Google Assistant searches to generate visual responses, Abrams said, will allow developers to “build apps for entirely new Assistant use cases, things that previously weren’t well suited to having voice-only interface, such as shopping for clothes or ordering food from a lengthy menu.”

This new feature is expected out later this year. The Actions on Google platform for the creation of voice apps will be available outside the U.S. later this year as it expands to France, Canada, Australia, Germany, and Japan.

Google Lens for computer vision in the world

Virtually all emphasis for Google Assistant in its first year was on voice as an interface, but with the new Lens service launch today, Google will bring computer vision to its assistant.

Open the camera with Google Assistant and point at a business, flower, or object, and Lens will attempt to identify what it’s looking at.

When held up to a theatre marquee during the keynote, for example, Lens on Google Assistant was able to recognize the words on the marquee and then tell you how to buy a ticket or add the event to your calendar. If you take a screenshot of a phone number, just tap it inside Google Assistant to make a call.

Lens will  be available initially for Google Assistant and Google Photos and will be integrated into other Google products in the future.

Google Assistant actions for Android and iPhone

Until today, Google Assistant actions could only be interacted with on Google Home. As was mentioned earlier, the Google Assistant third-party ecosystem of voice apps is still relatively new, with roughly 250 actions. That’s a far cry from the more than 13,000 in the Alexa Skills Store.

All devices that use the Google Assistant software development kit (SDK) will be able to speak with Google Assistant actions in the future, Abrams said.

Native payments for Google Assistant actions

During the developer keynote address Wednesday, Abrams said, “For payments you can choose to use Google facilitated payments, which are easy to integrate, allow you to leverage hundreds of millions of cards that users have already stored with Google, and are available to developers for free, or you can choose to use a payment method that users have already provided you.”

Payments on Google Assistant will have a dedicated area to allow users to track orders, modify an order, or place a new order.

Peer-to-peer payments are also on the way for Google Assistant, so you can send a friend or family money, not just businesses.

Google is getting smarter

News was made in both speech recognition and tensor processing units today, two areas related to Google Assistant’s AI smarts.

TPUs have facilitated AI at Google for a range of services, from search queries to the new smart reply feature. The second-generation tensor processing unit, or Cloud TPU, will reduce the amount of time necessary to train or run AI models from weeks to a matter of days or hours, AI leaders at Google said.

The word error rate for speech recognition systems continues to improve, and Google’s speech recognition now enjoys a 4.9 percent error rate. That’s below the historic lows Microsoft reported last year.

Software development kit partners

During the keynote address Wednesday, a list of companies intending to use the software development kit to place Google Assistant in their devices was briefly mentioned. Take a look at the slide below and you may see some new names of companies that intend to create their own devices with Google Assistant inside.

The Google Assistant SDK was made available to developers for the first time last month.

Smart home control for LG and GE appliances

LG Electronics and GE Appliances can now be controlled by Google Assistant. The two join dozens of smart appliance offerings made available through Google Assistant. Integrations for August Home Lock, LIFX smart lightbulbs, and other popular smart home appliances were added in March.