Problem ingesting vpc flow logs

Elastic Stack Beats
1

We are using filebeat 7.6.2 to ingest vpc flow logs. IT appears that it is taking longer then 24 hours for the vpc flow logs to be ingested.
Everything seems to be working fine for the elb logs. The SQS queue still has messages from noon yesterday in it. The number of messages available does fluctuate. Is this a known issue? How can we rectify this?

2

Is the aws vpc flow log supported?

3

I'm also facing the same issue, i'm getting error like "ERROR [s3] s3/input.go:475 ReadString failed: context deadline exceede" and SQS queue is full , it's unable to process the messages. I have increased visibility timeout to 45min but still getting the error.

4

I have been advised to upgrade to the latest filebeats and use IAM roles that should resolve the issue

5

I'm using latest 7.8 version, you're issue resolved? and what is IAM Role configuration?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.