Amazon S3 is a popular platform for object storage. Through the use of APIs, though, storage vendors can offer their own take on S3, potentially improving management and cost.
Technology generates, consumes and stores a massive amount of data -- dozens of zettabytes. Data management, analysis and use are more challenging than ever.
One technology that's emerged to help manage data is object storage. Amazon S3 launched more than a decade ago and made object the standard for cloud data storage. It's a flexible, secure and cost-effective way to store, manage and use data, no matter the type or usage. But users don't have to go directly to Amazon to take advantage because several vendors offer S3-compatible storage.
Object storage is an ideal way to store data in the cloud because it's unstructured, defining data as an object with metadata and a unique identifier to locate and access each data unit. Most S3 data stores enable custom metadata for objects so users can manage them more finely.
What does 'S3-compatible' mean?
S3 is the largest and most popular public cloud storage service, so the tech world has embraced it for interface development and applications that use data. The term S3-compatible means the data store uses the S3 API to communicate with any connected device or system. It originally referred to data stored in a public cloud; however, S3-compatible storage has extended to on-premises and private cloud deployments.
Systems, devices and applications that use the S3 API should be able to connect easily with S3-compatible storage deployments, regardless of location or use.
How does S3 storage work?
S3 stores data as objects within buckets, which are containers for objects. Each bucket is typically stored in a designated region, whether in a public cloud or a private cloud within a single data center facility. Data does not leave the region unless specifically moved or replicated in another region. This structure helps customers with latency, cost and compliance.
Systems, devices and applications that use the S3 API should be able to connect easily with S3-compatible storage deployments, regardless of location or use.
Users can typically store any number of objects in a bucket, but many S3 data stores limit the number of buckets per account. Bucket policies enable access control to the data for management of systems, applications and users. They can add or deny permissions to objects based on specific criteria, such as requester, S3 actions and IP address.
A key is the unique identifier of an object within a bucket. Each object in a bucket has one key, which is combined with the object metadata and bucket information to create a unique object identifier.
Depending on the S3 storage product, users can also enable object version IDs to preserve, retrieve and restore every version of every object stored in buckets.
Top S3-compatible products and vendors
Vendors offer a range of S3-compatible products and services:
Backblaze has S3-compatible APIs so customers who use anything that supports S3 can point that data to the vendor's B2 Cloud Storage.
Cloudian's HyperStore offers an S3-compatible storage option that deploys in various ways, including on-premises, private and hybrid cloud.
Cohesity provides a suite for S3-compatible object data storage and management at scale.
HPE GreenLake for Scality and Storage offers enterprise-level data management with hybrid, cloud-ready, S3-compatible object stores.
MinIO offers S3-compatible object storage through its open source, Kubernetes-based platform. This structure opens up to public, private and edge clouds, no matter the Kubernetes distribution in use.
Scality Ring and Artesca provide scale-out support of the S3 API, with AWS-compatible authentication. Other features include S3 object locking for immutability, and hybrid cloud replication and lifecycle management.
Wasabi hot cloud storage has an S3-compliant interface for S3-compatible storage applications, gateways and other platforms, according to the vendor. It integrates with any application or system that typically uses Amazon S3.
