Duplicate entries result from multiple Winlogbeats



I have multiple installs of Winlogbeat clients pushing, amongst other logs, events for locked out users. All works as expected. The only problem we are now faced with is that as there are multiple instances of WLB on different servers, all pointing at the same ES index, I'm finding that as the event gets logged on the first DC, that event is logged by WLB and sent to ES. The event is then synced to the PDC by AD, lands on the PDC, the WLB fires on the PDC emulator and a new event is created by WLB and fired at the ES server. This creates two entries in the index of essentially the same event (albeit different record numbers as they came from different sources).

This produces inaccurate results in the Kibana reports. My current thinking is that I could use something similar to document_id but as each record is different, the only common field is something like the SID of the user. Using this solution wipes out any historical data.

Any pointers appreciated.

(Andrew Kroh) #2

Can you please share the JSON content of the two "duplicate" events. You can copy the JSON out of Kibana's Discover page. It will be easier to offer advice if we can see the data in question. Feel free to anonymize the data as necessary.

(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.