Filebeat loses or doesn't deliver messages from AWS CloudTrail

Hello team.
I installed and configured filebeat to collect logs in aws.
I use the aws module to collect logs cloudtrail.
The data began to be collected, but I noticed that the data was not included in the log - "eventName": "CreateVolume"

All messages before and after it are available, but this event itself does not exist. The cloud itself contains information about this event. I checked several times - the result is the same.
Tell me what could be the reason? Is this a module problem?
I use a standalone cluster elk.
From the logstash side, the filtering is primitive, which does not concern this event.

out of 100 messages only 86 were delivered. 14 were lost...

I checked on another message and it turns out there are still notifications that do not get into the elastic - "eventName": "DescribeAlarms". Moreover, not all messages, otherwise only some of them are not received.
It turns out that filebeat is not working correctly and does not pick up all messages from the queue/file in the cloud?

Or is this normal behavior when filebeat loses messages randomly?

As it turned out, the problem was elasticsearch. problem when matching dynamic field type.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.