Google is bringing tensor processing units to the edge. At the Google Cloud Next conference in San Francisco, the company introduced Edge TPU, an application-specific integrated circuit designed to run TensorFlow Lite machine learning models on mobile and embedded devices.
The announcement is indicative of both the red-hot AI hardware market, as well as the growing influence machine learning is having on the internet of things and wireless devices. But the Edge TPU also gives Google a more comprehensive edge-to-cloud AI stack to compete against the likes of Microsoft, Amazon and IBM as it looks to attract a new generation of application developers.
Analysts called the move a good one. "This fills in a big gap that Google had," said Forrester's Mike Gualtieri.
Spotlight on model inferencing
Google's cloud environment is a fertile ground for training AI models, a nontrivial process that requires enormous amounts of data and processing power. Once a model has been trained, it's put into production where it performs what's known as inferencing, or where it uses its training to make predictions.
Mike Gualtieri
A growing trend is to push inferencing out to edge devices such as wireless thermostats or smart parking meters that don't need a lot of power or even connectivity to the cloud, according to David Schatsky, managing director at Deloitte LLP. "These applications will avoid the latency that can be present when shuttling data back and forth to the cloud because they'll be able to perform inferencing locally and on the device," he said.
But Google customers who wanted to embed their models into edge devices had to turn to another provider -- Nvidia or Intel -- for that kind of functionality. Until now. The Edge TPU will give Google customers a more seamless environment to train machine learning models in its cloud and then deploy them into production at the edge.
David Schatsky
It also appears to be a nod to the burgeoning relationship between AI and IoT. According to Schatsky, venture capital funding in AI-focused IoT startups outpaced funding to IoT startups overall last year. "AI is so useful in deriving insight from IoT data that it may soon become rare to find an IoT application that doesn't use AI," he said.
A competitive stack
They're not just saying this is a TPU and you can run it on the edge. No, they're saying this is a fundamentally new chip designed specifically for inferencing.
Mike Gualtierianalyst, Forrester
The Edge TPU is in the same vein as an announcement Microsoft made last year with Project Brainwave, a deep learning platform that converts trained models to run more efficiently on Intel's Field-Programmable Gate Arrays than on GPUs, according to Gualtieri. "There is a fundamental difference in training a model versus inferencing a model," he said. "Google recognizes this. They're not just saying this is a TPU and you can run it on the edge. No, they're saying this is a fundamentally new chip designed specifically for inferencing."
Indeed, Gualtieri said, the Edge TPU makes Google more competitive with Microsoft, Amazon and even IBM, all of which made moves to differentiate between model training and model inferencing sooner. "This is an effort, I believe, for Google to make its cloud more attractive, oddly by saying, well, yes, we have the cloud, but we also have the edge -- the non-cloud," he said.
James Kobielus, lead analyst at SiliconAngle Wikibon, also sees the Edge TPU as a strategic move. He called the Edge TPU an example of how the internet giant is creating a complete AI stack of hardware, software and tools for its customers while adding a force multiplier to compete against other vendors in the space.
James Kobielus
"Google is making a strong play to build a comprehensive application development and services environment in the cloud to reach out to partners, developers and so forth to give them the tools they need to build the new generation of apps," he said.
Kobielus highlighted the introduction of the Edge TPU software development kit as another example of how Google is planning to compete. The dev kit, which is still in beta and available to only those who apply for access, shows a "great effort" to convince developers to build their apps on the Google cloud and to catch up to Amazon and Microsoft, both of which have a strong developer orientation, he said. "They needed to do this -- to reach out to the developer market now while the iron is hot," he said.
What is the Google AI stack missing? It's too soon to tell, both Kobielus and Gualtieri said. But with innovation in AI happening at breakneck speed, companies should see this as a part of an evolution and not an end point.
"Different applications are going to require even different chips," Gualtieri said. "Google is not behind on this. It's just what's going to happen because there may be very data-heavy applications or power requirements on smaller devices. So I would expect a whole bunch of different chips to come out. Is that a gap? I would say no because of maturity in this industry."
Mike Gualtieri
David Schatsky
James Kobielus
