We're using version 1.0.1 of the logstash-input-cloudwatch_logs logstash plugin and are randomly encountering mysterious gaps in the CloudWatch log events. We periodically have an event that is clearly in CloudWatch that never appears in ElasticSearch and another instance of the same event later does get fetched by the plugin and appears in ES. Basically, we can trigger the same event to CloudWatch over and over and the plugin retrieves them most of the time, but randomly misses an event. The symptoms suggest there is some timing issue - possibly the missed event is coming into CloudWatch as the logstash-input-cloudwatch_logs plugin is starting to poll. There's no evidence of any errors in the LogStash log file.
I've reviewed the code for the plugin and searched the bug list since v 1.0.1 to see whether this is a bug in the version we're using, but haven't found the issue reported anywhere. Not sure how to debug it.