Investors share their predictions for AI and machine learning in 2018

Over the past three years, building intelligent apps — apps powered by machine learning that continuously improve as they ingest new data — has become easier and easier. Given the continued rise of machine learning, where are venture capitalists looking for the next set of investment opportunities? Generally, we see the core machine learning tools and building block services maturing, and now we are most interested in companies that are “moving up” the stack toward vertical applications, “moving down” the stack toward purpose-built hardware, and “moving out” of the data center toward intelligence at the edge.

Here are four categories which we have been investing in and tracking closely because we believe they will play meaningful parts in the future of intelligent apps.

1. Moving out: Intelligence on the edge

As machine learning becomes more prevalent, cloud providers have raced to offer the latest GPUs for training machine learning models. This growth in demand for GPUs is readily apparent in Nvidia’s financial performance, which shows their data center business nearly tripling between 2017 and 2018. However, the data center numbers only tell part of the story in the world of machine learning.

After machine learning models are trained on GPUs at a data center, companies often want to run these trained models at the edge. For example, a smart speaker may want to process portions of its audio recordings locally (think “Alexa,” “Hey Siri,” or “OK Google”) in order to reduce power consumption, ensure privacy needs are met, reduce latency, and lower bandwidth consumption.

The problem of efficiently bringing intelligence to the edge is difficult to solve because many of our current machine learning approaches were developed for expensive, always-on, always-connected hardware. Instead of adopting these approaches, the most interesting startups are working with cheap, power-constrained, and limited connectivity devices.

The companies we see working on this problem are taking two different approaches: One is an emphasis on creating better software and algorithms that can run models more efficiently on resource-constrained devices, and the other is creating more specialized combinations of software and hardware to run models more efficiently. We think there is merit in both approaches, and understanding customer use cases will be the key to building the right solutions.

2. Moving down: Purpose-built AI hardware

Purpose-built hardware at the edge or core is becoming increasingly important in the field of machine learning because companies can run software much more efficiently if they use specialized chips.

The most striking illustration of this trend has been in the world of Bitcoin mining. Bitcoin mining started with CPUs in 2008 and 2009, and moved to GPUs, then FPGAs, and now ASICs, which are specialized chips that can only perform one operation. This hardware evolution clearly demonstrated that special-purpose chips can perform orders of magnitude better than general purpose hardware such as a CPU.

 

We are seeing this same trend in the world of machine learning. As Nvidia continues to optimize its GPUs for deep learning, Azure and AWS released FPGA instances that can be customized for specific workloads and Google released the TPU, an ASIC optimized for machine learning.

The key question in this world of specialized hardware is the upfront cost in time and capital to design, build, and operate specialized software and hardware for a particular workload, relative to the improvement in ongoing performance and cost of running the same workload on general purpose hardware such as a CPU or GPU. We think about this tradeoff in the context of training models (in a data center), inference in a data center, and inference at the edge.

For example, training “at the core” in a data center will likely be the predominant mode of training models, so the cloud providers will have a strong incentive to build special-purpose hardware in order to improve performance, reduce their reliance on suppliers, and have a higher level of control on margins. The cloud providers will also be best equipped with the data and expertise to do the cost-benefit analyses on building new chips for specific workloads.

Inference “at the edge” will also likely have many high-value and high-volume use cases, such as object avoidance in autonomous vehicles, emergency systems for industrial equipment, and voice recognition for smart home devices. For many of these use cases, it will be important to combine software and hardware to get the best mix of performance and price. There is opportunity here for startups as the bigger players chase the core and build generalized systems for the edge.

3. Moving out: Natural user interfaces

Another key component of intelligent apps will be the continued improvement in natural user interfaces — any interface that we use to communicate with other humans including text, voice, vision, gestures, and other forms of body language.

Consumers today have become significantly more comfortable talking to their devices, with over 35 million U.S. consumers using voice-activated speakers. Nevertheless, it is still very difficult for the current generation of digital assistants to understand what we really mean, and there have been many examples of funny misunderstandings between Alexa and her human users.

Two of the primary approaches we have seen in startups tackling the problem of improving understanding and user experience is to either narrow down the potential universe of requests and responses to a particular use case or to include humans in the loop to augment the machine learning system.

For example, scheduling bots have become some of the most widely used AI assistants. These bots have a (relatively) constrained set of text-based inputs that lead to a finite number of outputs: Send a meeting invite, update an invite, cancel an invite, or ask for more information. This allows them to better anticipate what users are asking for and create a more humanlike interaction model for their users. Injecting a human into this loop to audit the machine learning predictions can also drastically improve the user experience by ensuring that all responses make sense before they are sent out, leading to more trust in the system.

4. Moving up: Vertical AI applications

Lastly, vertical applications that have the ability to become intelligent continue to be a major focus for our firm’s investments.

As intelligent apps become easier to build, we see teams that deeply understand customer pain points and build solutions to alleviate those pain points. We have seen this in everything from procurement to administering health care plans. We believe that the most valuable insights will come from companies that have a strong understanding of their customers, because while there will be many tools which can help surface insights from data, exposing the right insights through the right user interface and inserting that into the current workflow to add value will become the harder problem.

There is a lot of opportunity for AI and machine learning, and we want to back entrepreneurs who believe that every successful application of the next decade will be intelligent.

S. Somasegar is the managing director of Madrona Venture Group, a venture capital firm that teams with technology entrepreneurs to nurture ideas from startup to market success.

Daniel Li is a principal at Madrona Venture Group.