At the 2013 Amazon Web Services re: Invent conference, Amazon announced the launch of Amazon Kinesis, its service for real-time big data processing. Just recently, the technology has become broadly available for use.
Terry Hanold, AWS vice president of new business initiatives, called the product a response to a shortcoming in Hadoop and other technologies that process data in batches. While those technologies are capable of handling large quantities of data, Amazon says, they don't enable real-time, ongoing decision making on data that's constantly streaming. Amazon Kinesis will provide such service on AWS' standard pay-as-you-go basis.
Following are answers to frequently asked questions about Amazon Kinesis to help you decide if it's a fit for your goals.
What is Amazon Kinesis?
Amazon Kinesis is a managed service for real-time processing of streaming data on a large scale. It helps build applications that respond quickly to changes in a data stream.
Kinesis applications allow users to make in-the-moment operational decisions.
How can Amazon Kinesis be used?
Kinesis is ultimately meant to simplify the process of building apps that operate on real-time data. According to Amazon, it's capable of collecting and processing hundreds of terabytes per hour from hundreds of thousands of sources. In other words, it helps manage information that streams in constantly and at high volume. That category can include anything from operating logs to financial transactions, from social media feeds to clickstreams.
Kinesis applications allow users to make in-the-moment operational decisions, such as building real-time dashboards; generating alerts; and sending data to other big data services, such as Amazon Redshift or Amazon Dynamo Database (DynamoDB). Because the streamed data is stored in several AWS availability zones for 24 hours, users have a window of time in which to read and analyze the data, as well as move it into long-term storage.
Who should be using Amazon Kinesis?
Generally speaking, the service is intended for developers building real-time applications, particularly those dealing with large volumes of data. James Staten, a Forrester Research analyst, has said that Kinesis would be particularly useful for companies such as Autodesk Inc., which is looking to bring computer-aided design (CAD) apps and three-dimensional rendering tools to the cloud. Amazon also specified the types of data that might be in an Amazon Kinesis use case: IT infrastructure log data, social media, game development, Web clickstream data, application logos and market data feeds.
What benefits does Amazon Kinesis offer?
Amazon Kinesis integrates with Redshift, Dynamo DB and Amazon Simple Storage Service, or (S3, as well as third-party products, a perk for developers who don't want to abandon their personal data-processing methods. As to security, Kinesis authenticates users through cryptographic methods that prevent unauthorized parties from accessing data.
What's the pricing model for Amazon Kinesis?
Like other AWS services, Kinesis is priced on a pay-as-you-go basis. Users are billed based on the amount of data processed and the way in which it's packaged. For further information, the Amazon Kinesis webpage goes more in-depth.
How do I get started?
You can generate an Amazon Kinesis stream in one of two ways: You can go through the AWS Management Console or you can make an API request. Unsure how? Amazon will walk you through it.
