Getty Images/iStockphoto
Be aware of these CloudWatch Logs limits and quotas
CloudWatch Logs imposes certain limits and service quotas related to log data ingestion and other areas. Fortunately, in many cases, there are workarounds.
CloudWatch Logs is an essential tool to implement operational best practices in AWS. The service provides storage, analysis and processing capabilities for log data from multiple sources, including on-premises servers and EC2 instances.
While it's a useful and scalable service, CloudWatch Logs has some intrinsic limits. These limits are related to concurrency, throttling, log data ingestion, timeouts and field parsing. Admins should note these limits before using CloudWatch Logs, and, when possible, work around them.
Log data limitations
To export log data into CloudWatch Logs, applications call the PutLogEvents API, which uploads an array of log events, as a batch, into a log stream. The size of the batch is based on the number and size of submitted log events. Each log event can be a maximum size of 256 KB, and the total batch size can be a maximum of 1 MB. Since these limits cannot be increased, closely monitor sizes when sending log events into a log stream.
The CloudWatch Logs agent is a common and recommended method for exporting logs. The agent skips log events that exceed 256 KB in size. The agent also automatically adjusts the length of each batch to comply with the 1 MB restriction.
For high-volume applications, or tasks that require exporting a large amount of data into CloudWatch Logs, the PutLogEvents API has a limit of 5 requests per second per log stream. Even though this limit -- which cannot be increased -- is enough for most tasks, there are situations where a log export job can reach it. Create multiple log streams and ingest data concurrently to increase the throughput. In doing this, you avoid the limit and achieve the desired throughput.
Timeout and query limitations in CloudWatch Logs Insights
CloudWatch Logs Insights is a useful tool for troubleshooting, as well as extracting operational data. It enables developers to analyze and aggregate log data using a predefined query language. A limitation to CloudWatch Logs Insights, however, is that queries time out after 15 minutes.
To avoid this, narrow the timestamp range to reduce the data window being analyzed. Alternatively, split the data into multiple log streams, so that queries cover smaller amounts of data and execute below the timeout limit. Then, add up the results from multiple query executions to cover the desired data range.
CloudWatch Logs Insights supports querying multiple log groups simultaneously. Although this is a useful feature to analyze log data, developers can access only up to 20 log groups in a single query -- a quota that cannot be increased.
If an application needs to query more than 20 log groups, split queries and then aggregate data from multiple queries. CloudWatch Logs Insights supports a maximum of 10 concurrent queries.
Stay up to date on service quotas
The CloudWatch Logs quotas mentioned in this article are some of the most relevant to cloud admins as of the date of publication. Reference AWS documentation for the full and up-to-date list of service quotas to avoid potential blocking issues when building and operating scalable cloud applications.
Limitations on filters
Metric filters are a CloudWatch Logs feature that let developers parse text in log data and convert patterns into CloudWatch metrics. This is a convenient way to monitor application events and automate actions such as notifications, alarms or custom logic. There is a limit of 100 metric filters per log group, which cannot be increased. This limit should be enough to implement a wide range of filters as a result of log events. Otherwise, it's possible to split data into multiple log groups to configure a higher number of metric filters.
Subscription filters allow incoming log data to integrate with other AWS services, such as Lambda, Kinesis Firehose or Kinesis Data Streams. There is a fixed quota that limits each log group to a maximum of two subscription filters. One way to work around this is to use Kinesis Streams or Lambda functions as an interface to integrate with more AWS services, beyond the quota in CloudWatch. This would allow log data to be exported to any number of AWS services.
API throttling
There are also throttling limits in CloudWatch Logs APIs, including the following:
- CreateLogGroup
- CreateLogStream
- DeleteLogGroup
- DescribeLogGroups
- DescribeLogStreams
- FilterLogEvents
- GetLogEvents
- GetLogEvents
- PutLogEvents
Each API has its own limit in terms of requests per second, which also varies by region.
