Google's Cloud TPU v2, v3 Pods accelerate ML training

First of the tech giants

While Amazon and Facebook have been working on AI chips of their own -- albeit ones focused more on inference -- Google is the first of the tech giants to make such a processor publicly available, said Peter Rutten, research director at IDC.

Many startups are working on AI-centric chips as well, Rutten said, but few have gone to market yet.

"Apart from what the other vendors are planning, the Google TPU Pods appear to be extremely powerful," Rutten said.

Apart from what the other vendors are planning, the Google TPU Pods appear to be extremely powerful.

In general, the benefits of application-specific integrated circuits (ASICs), such as Google's TPUs, are their speed, low energy use and low unit cost, Rutten explained. However, ASICs are fixed and cannot be adjusted as AI algorithms change.

"If Google keeps delivering new versions at the speed that they have been until now, though, that may not be a problem," Rutten said. "Bottom line: Data scientists are getting a lot of performance for AI model training from this offering."

Google Cloud TPUs represent a faster and more cost-effective way to handle large machine learning workloads, Pete Voss, a Google spokesman for cloud AI and machine learning, said in a phone interview.

"Developers can iterate in minutes and train large production models in hours instead of days," Voss said.

A variety of customers, including eBay, Recursion Pharmaceuticals, Lyft and Two Sigma, use Cloud TPU products, Voss said.

In a recent case study, Recursion Pharmaceuticals dramatically lowered its training time for a model that iteratively tests the viability of synthesized molecules to treat rare illnesses, Voss said. It took the company more than 24 hours to train the model on an on-premises cluster and 15 minutes to train it on a Cloud TPU Pod, he said.

The price of machine learning power

Both the Cloud TPU v2 and v3 Pods primarily perform the same functions, providing users with shorter routes to insights, higher model training accuracy and the ability to do frequent model retraining, according to Google. Cloud TPU v3, the more expensive service option, features upgraded hardware and provides faster results.

Alone, both Cloud TPU v2 and v3 devices price in the single digits per hour. The Cloud TPU v2 Pod, meanwhile, ranges from $24 an hour to rent a 16 TPU chip pod slice to a little under $400 an hour to rent the entire Pod, with two in-between options.

The Cloud TPU v3 Pod is priced at $32 an hour for a 16 TPU chip pod slice. Users may also pay to rent any of the pod slice options by the year or by three years, for hundreds of thousands of dollars.

Rutten said the accelerated server market, while currently dominated by GPUs, will start to see more and more AI ASICs over time.

Yet, he said, GPUs shouldn't be eliminated or ruled out.

"There are a lot of benefits with GPUs: their flexibility, programmability and, most of all, the software stack -- think CUDA -- and ecosystem around them," he said. "So, don't just look at the benchmarking results when comparing AI processors."