I have packetbeat installed in all the EC2 instances. The ignore_outgoing is set to 'True' which prevents the duplicated transactions being stored in the ES.
The applications makes call to MongoDB,as the ignore_outgoing is true, the failed transactions to MongoDB are ignored. I don't see any Error MongoDB transcations in Kibana.
I thought to install the packetbeat in a dedicated server, but it requires port mirror in switch, which is not feasible in AWS.
Any ideas why don't we introduce 'ignore_incoming'?
What's the exact use-case for 'ignore_incoming' ? If you have packetbeat installed on all servers, the server doing the MongoDB request should forward the failed one, right?
Have you tried if the failed MongoDB transactions are sent to ES if ignore_outgoing is set to false?
There are MongoDB timeout in the applications logs, but I don't see any failed MongoDB transactions in the Kibana.
I will try 'ignore_outgoing=false', will let you know the result.
About MongoDB we've found some issues in message correlation. Follow this ticket for current state: https://github.com/elastic/packetbeat/issues/377 . A fix is proposed in this PR: https://github.com/elastic/packetbeat/pull/405