When I use the aws-cloudwatch input plug-in to obtain the logs stored in cloudwatchlogs, the number of logs obtained is inconsistent with the number of logs in cloudwatchlogs. The route53 logs are stored in cloudwatchlogs. The number of logs is relatively large.
For example, in the past two hours, there were 140W logs in cloudwatchlogs, but only 130W logs were pushed from filebeat to ES
- type: aws-cloudwatch
log_group_arn: arn:aws:logs:ap-northeast-1:xxxxx:log-group:xxxxx:*
scan_frequency: 30s
start_position: end
region_name: ap-northeast-1
access_key_id: xxxx
secret_access_key: xxxxx
fields:
env: xxx
fields_under_root: true
latency: 1m