Amazon Web Services has announced that Amazon Kinesis, a fully managed service for real-time processing of high volume, streaming data, is now broadly available for use.
Using Amazon Kinesis, a customer can store and process terabytes of data an hour from hundreds of thousands of sources, making it easy to write applications that take action on real-time data such as web site click-streams, marketing and financial transactions, social media feeds, logs and metering data, and location-tracking events.
Amazon Kinesis-enabled applications can power real-time dashboards, generate alerts, drive real-time business decisions such as changing pricing and advertising strategies, or send data to other big data services such as Amazon Simple Storage Service (Amazon S3), Amazon Elastic Map Reduce (Amazon EMR), or Amazon Redshift. Customers can create an Amazon Kinesis stream with a few clicks in the AWS Management Console or through a simple API call.
To date, most big data processing has been done through batch-oriented approaches such as those used by Hadoop, or through database technologies such as data warehouses. Since these systems store and process data in batches, they aren’t able to support applications that need to process streaming feeds of rapidly changing data in real time. To build applications that rely on this “fast-moving” data, many companies have developed their own systems or stitched together open source tools, but these are often complex to build, difficult to operate, inelastic and hard to scale, and can be unreliable or lose data, according to an AWS release.
Amazon Kinesis claims to help solve these problems by providing a fully managed service that takes care of all the “heavy-lifting” for developers, providing easy data ingestion and storage, high data durability, and the ability to scale seamlessly from kilobytes to terabytes an hour – at a fraction of the cost and effort.
The Amazon Kinesis client library simplifies load balancing, coordination, and fault tolerance, and developers can use AWS Auto Scaling to easily create elastic, high-performance Amazon Elastic Compute Cloud (Amazon EC2) processing clusters. Amazon Kinesis also integrates with third-party products, giving developers control and freedom to choose their preferred method of data processing, including popular open source products.
With Amazon Kinesis, customers can quickly and easily add real-time analytics and other functionality to their applications, claims the release, turning today’s explosive data growth into an opportunity to build competitive advantage and innovate for their customers. Amazon Kinesis is currently available in the US East (N. Virginia) Region and will be rolled out to other AWS Regions in the coming months.