# Duplicate entries with Winlogbeat with WEF

**URL:** https://discuss.elastic.co/t/duplicate-entries-with-winlogbeat-with-wef/231909
**Category:** Beats
**Tags:** winlogbeat
**Created:** [May 10, 2020, 8:56pm UTC](https://discuss.elastic.co/t/duplicate-entries-with-winlogbeat-with-wef/231909 "2020-05-10T20:56:29Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![NightSpark](https://avatars.discourse-cdn.com/v4/letter/n/ed8c4c/32.png) [@NightSpark](https://discuss.elastic.co/u/NightSpark)
#### Post date: [May 10, 2020, 8:56pm UTC](https://discuss.elastic.co/t/duplicate-entries-with-winlogbeat-with-wef/231909/1 "2020-05-10T20:56:29Z")

</div>

Hi

New to elasticsearch and have just setup Winlogbeat on a central Windows 2008 R2 server which is collecting windows events from machines using Windows Event Forwarding  
I am using V 7.6.2 of elasticsearch and winlogbeat

I am finding that when I go into Kibana and into discover that view events that there are duplicate entries and that this keeps increasing over time. By duplicate I mean they have a different \_id but everything else is the same, i.e timestamp, winlog event info such as id and computername. In approx 9 hours I have 60 duplicates of each entry.  
I have had play around with using fingerprint processor settings and document\_id in output with no luck.  
My winlogbeat.yml is as follows

```auto
// 
winlogbeat.event_logs:
  - name: ForwardedEvents
    ignore_older: 72h
    
setup.template.settings:
  index.number_of_shards: 1

output.elasticsearch:
  hosts: ["elastic:9200"]
  ssl.certificate_authorities: ["C:/Program Files/Winlogbeat/elasticsearch-ca.pem"]
  username: "redacted"
  password: "redacted"
  protocol: "https"

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
//

```

---

<div class="post-metadata">

### Author: ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)
#### Post date: [May 11, 2020, 9:29am UTC](https://discuss.elastic.co/t/duplicate-entries-with-winlogbeat-with-wef/231909/2 "2020-05-11T09:29:24Z")

</div>

Hey @NightSpark, welcome to discuss 🙂

Unless this is an unknown bug, there are some reasons this may happen:

- Multiple instances of Winlogbeat are running.
- Connectivity issues cause events to be sent again.

Could you please check if you have multiple instances of Winlogbeat running in your system?

To discard connectivity issues, or other issues related with the output, you can check the logs, and you can try to temporaly use the [file output](https://www.elastic.co/guide/en/beats/winlogbeat/7.6/file-output.html) and check there if events are also duplicated.

---

<div class="post-metadata">

### Author: ![NightSpark](https://avatars.discourse-cdn.com/v4/letter/n/ed8c4c/32.png) [@NightSpark](https://discuss.elastic.co/u/NightSpark)
#### Post date: [May 13, 2020, 11:12am UTC](https://discuss.elastic.co/t/duplicate-entries-with-winlogbeat-with-wef/231909/3 "2020-05-13T11:12:02Z")

</div>

Thanks @jsoriano  
I confirmed that there is only one instance of Winlogbeat running.  
The file output was a good tip and I can see that there are duplicates in that output as well.  
I am investigating if the source has duplicates, i.e the collected windows events though WEF.

---

<div class="post-metadata">

### Author: ![NightSpark](https://avatars.discourse-cdn.com/v4/letter/n/ed8c4c/32.png) [@NightSpark](https://discuss.elastic.co/u/NightSpark)
#### Post date: [May 13, 2020, 10:42pm UTC](https://discuss.elastic.co/t/duplicate-entries-with-winlogbeat-with-wef/231909/4 "2020-05-13T22:42:35Z")

</div>

It is possible that subscription for forwarded events was including the collecting servers own forwarded events so it was in recursive loop.  
Restricting events to a subset has seemed to fix it.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 10, 2020, 10:57pm UTC](https://discuss.elastic.co/t/duplicate-entries-with-winlogbeat-with-wef/231909/5 "2020-06-10T22:57:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
