Seraphim Vector - Fotolia
Snowball Edge enhances data migration to AWS
Enterprises use Snowball to speed data migrations, but the portable appliance has its limitations. With its newer Edge devices, AWS added compute capabilities to address that.
Enterprises can get frustrated by the time it takes to replicate large data sets over a network, from on premises to the cloud, which is why AWS offers a portable storage appliance to accelerate the process.
AWS rolled out Snowball devices two years ago amid growing enterprise cloud adoption. Earlier this year, the provider added Snowball Edge to incorporate local processing into the shippable appliances. Snowball Edge should address some of the products' earlier limitations, but don't expect it to make EC2 a stalwart of your on-premises deployments.
Hybrid cloud usage, feasibility
At first glance, Snowball Edge might seem like a form of hybrid cloud infrastructure. But its pricing model and documentation reaffirm that AWS designed it for data transfer and not ongoing application operations.
AWS incorporated EC2 instances into the appliance to support data schemas that require complicated extract, transform and load (ETL) operations or other forms of local processing before data can be stored in S3. Snowball Edge won't replace a local hyper-converged system or act as a permanent hybrid appendage to AWS infrastructure. Instead, organizations should look to the AWS-VMware partnership, where AWS has ported services like Amazon Relational Database Service to on-premises hardware and provided a consistent management interface for the two environments.
Snowball Edge is well-suited to run local compute jobs for remote locations and harsh environments, where high-speed network connections are unavailable, sporadic or expensive. Such scenarios are often found in manufacturing sites, oil and gas rigs, disaster recovery zones, and mining and construction sites. A user can run Lambda jobs triggered by S3 activity and let Snowball Edge handle ETL tasks, run management automation tasks or even perform analysis on incoming data streams.
Snowball Edge technical specifications
These upgraded Snowball devices double the storage capacity of the original -- up to 100 TB -- and come with a 1.8 GHz Intel Xeon D processor, with up to 24 virtual CPUs and 32 GB of memory. Snowball Edge also includes three network interfaces -- the original RJ45 and SFP+, along with a faster QSFP+ interface -- which require a compatible module and connector that can support 25 Gbps or 40 Gbps.
The multicore processor and other hardware improvements facilitate additional software features, including the following:
- an endpoint compatible with S3 or network file system for network file transfers via AWS S3 APIs or a standard network-attached storage protocol;
- an S3 Adapter for programmatic access to S3 buckets on an Edge device using S3 REST APIs;
- an SBE1 instance type to run compute instances booted from Amazon Machine Images (AMIs) -- the SBE1 instances can run up to 10 AMIs per device on one of six different instance types;
- Lambda functions written in Python with AWS Greengrass -- a user can trigger functions with S3 actions, such as a PUT object command;
- programmatic access to a subset of EC2 APIs, which can run compute instances on Snowball Edge;
- the ability to cluster five to 10 Edge devices to act as a single, scalable, redundant storage and compute pool to increase local storage, compute capacity and reliability. In cluster mode, the available storage is 45 TB per node. Amazon claims 99.999% durability for clustered data; and
- access to AWS tools and APIs to manage and monitor the service.
SBE1 instances run a subset of Linux AMIs, including Ubuntu Server 14.04 LTS (HVM), Ubuntu Server 16.04 LTS - Xenial (HVM) and CentOS7 with Updates (HCM).
When clustered, individual nodes can be configured to run EC2 instances, and each node can have a different mix of SBE1 instance types and AMIs that are controlled via the AWS Management Console.
Amazon also implemented hardware and software safety features on the devices to protect against potential security breaches. The device is built with a tamper-resistant package and includes Trusted Platform Module chips to prevent unauthorized hardware modifications and cryptographically protect firmware, local software and data.
Snowball Edge encrypts data with 256-bit keys with AWS Key Management Service, so keys aren't stored on the device and are automatically erased from memory when power is cut. The Snowball firmware isolates EC2-based applications and Lambda functions on a secure storage partition and controls data access through S3 APIs. When an organization uses the S3 Adapter and APIs, they rely on the same secure interfaces and encryption the Snowball client software uses to copy data.
The Edge also supports the Snowball Job Management API to automate administrative tasks, such as system configuration, EC2 instance management and storage job creation. The API also allows developers to create custom applications and integrations to existing systems.
Editor’s note: After publication, AWS added two more Snowball Edge options that offer more compute power. Snowball Edge Compute Optimized and Snowball Edge Compute Optimized with GPU both include 42 TB of S3-compatible storage and 7.68 TB of NVMe SSD storage. These devices can power an instance that requires up to 52 vCPUs and 208 GiB of memory.
The original Snowball Edge has been renamed Snowball Edge Storage Optimized.
