Missing some fields in some document

Elastic Stack Beats
,
1

I am using Packetbeat version 7.12.1 to collect DNS data from network flows, then sending it to Logstash, and finally forwarding it to Elasticsearch. I am encountering an issue where certain documents in Elasticsearch are missing the fields event.start, event.duration, and client.bytes, while these fields are present in other documents.

Could you please help me understand the reasons behind this and provide guidance on how to resolve it? Thank you very much

2

There could be several reasons for this. One thing to check is if there is another event for the same DNS transaction (dns.id) between the same source/destination that is complete. Sometimes a client will broadcast the same request more than once. Packetbeat has to associate the two sides of the transaction before generating an event. Duplicate requests can cause issues with that association algorithm. Packetbeat populates the notes field when it has issues like this.

1 Like
3

Thank you so much for your reply, I will check that information again

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.