AWS VPC Flow Log integration

Hi all,

We have been trying since a month to stabilize the integration of vpc flow logs using elastic agent.
There is always a lag of 2-3 days in the ingestion, the messages in the sqs queue never seems to do down because till the time a log file is processes by elastic agent another 2 files are received.
My cluster is made up of 3 hot nodes 2 of them having 10 CPU cores, 64 GB memory and high io ELB disks and the 3rd hot node having a 32 cores graviton CPU, 64 GB memory and same disk as above. We also have 2 dedicated ingest only nodes, and 2 warm nodes. Any tips on how to improve the ingestion performance and delay the log delay will be very helpful.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.