AWS ARM-based chips could shift microprocessor market
Amazon confronts cloud rivals with microprocessor designs. AWS ARM-based A1 instances offer high processing for lower costs and improved utilization.
Grappling with a tight budget, Emmanuel Santana was eager to trim a significant line item: his cloud-service fees. Santana is the CTO at Reamp, a São Paulo, Brazil-based company that gathers analytical insights from advertising data. The company's clients are large Latin American media companies that want to optimize their digital marketing campaigns.
Compute-intensive statistical analysis is a key piece of Reamp's business model. As cloud processing bills threatened his company's profit margin, Santana and his team were pleased to discover an affordable service in AWS' EC2 offering called A1. Designed to undercut mainstream cloud prices for certain tasks, A1 is powered by the AWS ARM-based microprocessor, Graviton1.
"We must keep costs down in order to be profitable," Santana said. "A1 instances came along at a good time a couple of years ago. We were able to save 20% over previous costs. Our application doesn't require that much memory, but it's very processing-intensive and repetitive."
As companies have come to depend on the cloud for more of their IT needs, organizations like Reamp have felt the pinch of high cloud costs, even as vendor lock-in increases. Smaller businesses might feel the most pain, as they are less likely than larger companies to earn volume discounts. For these businesses, a cloud service that costs anywhere from 10% to 40% less would be a godsend.
Another company that has adopted AWS A1 is SmugMug, a service that lets users upload, store and share photos and videos. SmugMug saved 40% over the cost of an Intel-based service for one of its applications, according to Shane Meyers, principal operations engineer at SmugMug. The company eliminated its entire data center in favor of AWS and uses an Amazon EC2 A1 instance for an application that serves photos to customers. "We moved photo serving to A1 with relatively little pain," Meyers said.
Although Reamp's and SmugMug's use of AWS A1 represents a small sliver of the cloud services market, it's emblematic of profound changes underway. Amazon, its cloud rivals and venture-backed startups are hard at work to develop more cost-effective and custom-tailored services based on ARM and other innovative microprocessors. These developments make only a tiny crack in the monolith of the x86 server market share but could nonetheless grow into a much larger fissure.
According to IDC, Intel dominated the $6.3 billion worldwide server microprocessor market in the fourth quarter of 2019, with 93.6% market share of $5.9 billion. Chips based on the x86 architecture, which include both Intel and AMD, comprised 98.9% of the server market. ARM-based architectures, in contrast, claimed only 0.4%. However, ARM-based designs grew at a rate of 125% compared with the fourth quarter of 2018.
"Beyond x86, I think there's room for another architecture, especially if it can scale across different applications, and ARM can certainly do that," said Shane Rau, an analyst at IDC, in Framingham, Mass. "Having an instruction set and an architecture that can scale from the low end up to supercomputers is a good thing."
Graviton1 and Graviton2
AWS introduced A1 utilizing the 64-bit ARM Neoverse cores of Graviton1 in 2018. The tech giant later introduced three offerings based on the significantly more powerful Graviton2 in December 2019. The Graviton chips are the work of Annapurna Labs, an Israeli firm that Amazon acquired in 2015. The three new EC2 instances based on Graviton2 are M6g for general-purpose workloads, C6g for compute-intensive tasks and R6g for workloads that process large data sets in memory.
The ARM architecture was originally created for low-power applications such as smartphone apps. When used in servers, AWS ARM-based chips can deliver significant compute power with relatively low power utilization. AWS calls this the best price performance in EC2.
Graviton1 and Graviton2 are under the umbrella of AWS Nitro, a multifaceted initiative to provide specialized EC2 services that are high performing and lower cost. A1 was originally available in virtualized instances that ran on the Nitro Hypervisor and was followed by A1 bare metal instances in October 2019. The Nitro system enables the incorporation of multiple microprocessor architectures, including AMD EPYC-based instances, which were introduced in 2018.
"We used to have Intel-based instances," said Ravi Murty, principal engineer, EC2 at AWS, in remarks at the AWS re:Invent conference in 2019. "We're now able to replace the processor and launch both ARM instances and AMD-based instances."
Cloud providers compete on AI-based services
Graviton1 and Graviton2 are not the only fruits of the acquisition of Annapurna Labs by AWS. AWS Inferentia is an Annapurna-developed microprocessor optimized for inference processing, in which trained machine-learning algorithms make predictions to influence decision logic. AWS has coupled Inferentia with Intel Xeon Scalable processors in its EC2 Inf1 instances. Inferentia is AWS' entry in a field that has attracted industry giants.
For example, Google has developed its Tensor Processing Unit, which accelerates machine-learning workloads. Microsoft has launched an Azure service that features AI acceleration provided by Graphcore processors. Facebook, meanwhile, is working on an AI chip of its own.
A threat to Intel?
How much could Amazon shake up the microprocessor market? Some experts see it as a kick in the pants to the open microprocessor market. "I don't see this as disruptive, but as stimulative," Rau said. "It could spur others to produce chips to meet these needs."
Alan Priestley
Analysts are less confident that Graviton could threaten Intel in the server arena. "There is constant speculation that ARM will disrupt the x86 market in the data center," said Alan Priestley, a Gartner analyst. "It has been discussed for 10 years, but it hasn't come to fruition." Priestley noted a number of unsuccessful attempts to capitalize on the virtues of ARM to capture significant server market share, including those by chipmakers Marvell, Qualcomm and AMD.
Despite that history, AWS is not alone in its march anew down the ARM path. Chinese tech-giant Huawei has developed an ARM-based microprocessor and related technologies. Microsoft has launched an ARM offering in Azure based on Marvell ThunderX2 microprocessors, and startups such as Ampere and NUVIA are trying their hand at new ARM designs. Meanwhile, startup Pensando has included ARM-based technology in its Distributed Services Card, which is poised to compete with AWS Nitro. These newcomers will face a large and established foe in AWS.
"Graviton is the only one that is in volume deployment in data centers -- it stands the most chance of getting ARM adopted in the data center, within the context of AWS services," Priestley said. Intel, however, is not standing still. The company said 50% of the chips it sells to cloud providers are custom chips, Priestley said.
Intel has optimized the chip for the need of the client, Priestly said, adding, "It's almost impossible to conceive that ARM could gain so much advantage over the x86 industry given the x86 industry has an interest in getting better."
Kevin Krewell
AWS itself could be the ultimate beneficiary of its heavy investment into Graviton designs. While AWS customers gain better price-performance for certain applications, AWS gains greater data center economy, which could deliver a competitive edge over time. "They [AWS] think in terms of how many data centers it saves," Priestley said.
Customers, meanwhile, could find themselves presented with a wealth of choices from competitors, as well as AWS' broad array of offerings for every need. "Amazon is going to be the one-stop shop for everything, so that nobody has to go anywhere else for anything -- like GM in cars," said Kevin Krewell, principal analyst at TIRIAS Research. "Even if it's a bit of a niche product -- if somebody needs it, they've got it."
That is, after all, how Reamp got started with A1, according to Santana. "We had a small team that wanted to experiment. They succeeded in streamlining their processes with this. It works and we keep using it," he said.
