# Duplicate events with Filebeat on windows on service restart

**URL:** <https://discuss.elastic.co/t/duplicate-events-with-filebeat-on-windows-on-service-restart/78743>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 15, 2017, 4:27pm UTC](https://discuss.elastic.co/t/duplicate-events-with-filebeat-on-windows-on-service-restart/78743 "2017-03-15T16:27:03Z")\
**Posts on this page:** 1\
**Showing post:** 1

<div class="post-metadata">

**Author:** ![eirc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eirc/32/11769_2.png) [@eirc](https://discuss.elastic.co/u/eirc)\
**Post date:** [March 15, 2017, 4:27pm UTC](https://discuss.elastic.co/t/duplicate-events-with-filebeat-on-windows-on-service-restart/78743/1 "2017-03-15T16:27:03Z")

</div>

We have a log file we are harvesting with Filebeat on some Windows hosts (2008 R2 Datacenter). Whenever the filebeat service is restarted the file gets fully resent.

Here the log of filebeat shutting down:

```
2017-03-15T18:16:10+02:00 INFO No non-zero metrics in the last 30s
2017-03-15T18:16:26+02:00 INFO Stopping filebeat
2017-03-15T18:16:26+02:00 INFO Stopping Crawler
2017-03-15T18:16:26+02:00 INFO Stopping 1 prospectors
2017-03-15T18:16:26+02:00 INFO Stopping Prospector
2017-03-15T18:16:26+02:00 INFO Prospector ticker stopped
2017-03-15T18:16:26+02:00 INFO Reader was closed: F:/wwwLogs/flights-wsapi/WebServicesWebApi.log. Closing.
2017-03-15T18:16:26+02:00 INFO Prospector channel stopped
2017-03-15T18:16:26+02:00 INFO Crawler stopped
2017-03-15T18:16:26+02:00 INFO Stopping spooler
2017-03-15T18:16:26+02:00 INFO Stopping Registrar
2017-03-15T18:16:26+02:00 INFO Ending Registrar
2017-03-15T18:16:26+02:00 INFO Total non-zero values: libbeat.logstash.publish.write_bytes=25069994 registar.states.current=1 registrar.writes=136 filebeat.harvester.started=1 libbeat.logstash.publish.write_errors=11 libbeat.logstash.published_but_not_acked_events=10424 libbeat.logstash.publish.read_errors=4 libbeat.publisher.published_events=272780 libbeat.logstash.publish.read_bytes=43641 publish.events=272781 registrar.states.update=272781 libbeat.logstash.call_count.PublishEvents=143 libbeat.logstash.published_and_acked_events=272780 filebeat.harvester.closed=1
2017-03-15T18:16:26+02:00 INFO Uptime: 3m15.984s
2017-03-15T18:16:26+02:00 INFO filebeat stopped.

```

and filebeat starting up:

```
2017-03-15T18:16:42+02:00 INFO Home path: [C:\Program Files\Filebeat] Config path: [C:\Program Files\Filebeat] Data path: [C:\\ProgramData\\filebeat] Logs path: [C:\Program Files\Filebeat\logs]
2017-03-15T18:16:42+02:00 INFO Setup Beat: filebeat; Version: 5.2.1
2017-03-15T18:16:42+02:00 INFO Max Retries set to: 3
2017-03-15T18:16:42+02:00 INFO Activated logstash as output plugin.
2017-03-15T18:16:42+02:00 INFO Publisher name: METADIAKOPES30
2017-03-15T18:16:42+02:00 INFO Flush Interval set to: 1s
2017-03-15T18:16:42+02:00 INFO Max Bulk Size set to: 2048
2017-03-15T18:16:42+02:00 INFO Additional config files are fetched from: C:/Program Files/Filebeat/conf.d/
2017-03-15T18:16:42+02:00 INFO Additional configs loaded from: C:\Program Files\Filebeat\conf.d\wsj.yml
2017-03-15T18:16:42+02:00 INFO filebeat start running.
2017-03-15T18:16:42+02:00 INFO Registry file set to: C:/Program Files/Filebeat/registry
2017-03-15T18:16:42+02:00 INFO Loading registrar data from C:/Program Files/Filebeat/registry
2017-03-15T18:16:42+02:00 INFO States Loaded from registrar: 1
2017-03-15T18:16:42+02:00 INFO Loading Prospectors: 1
2017-03-15T18:16:42+02:00 INFO Prospector with previous states loaded: 0
2017-03-15T18:16:42+02:00 INFO Starting Registrar
2017-03-15T18:16:42+02:00 INFO Start sending events to output
2017-03-15T18:16:42+02:00 INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2017-03-15T18:16:42+02:00 INFO Loading Prospectors completed. Number of prospectors: 1
2017-03-15T18:16:42+02:00 INFO All prospectors are initialised and running with 1 states to persist
2017-03-15T18:16:42+02:00 INFO Starting prospector of type: log
2017-03-15T18:16:42+02:00 INFO Harvester started for file: F:/wwwLogs/flights-wsapi/WebServicesWebApi.log

```

The registry file seems to reset the `offset` to 0 and thus resending the file. There's only one entry here and that looks ok - same `FileStateOS` data before and after the restart. But we also see a `registry.old` file when the service starts which looks like a copy of the registry file before the new start and the offset resetting.

```
[{
    "source": "F:/wwwLogs/flights-wsapi/WebServicesWebApi.log",
    "offset": 97601249,
    "FileStateOS": {
        "idxhi": 149225472,
        "idxlo": 510,
        "vol": 2488112230
    },
    "timestamp": "2017-03-15T18:19:08.1504+02:00",
    "ttl": -1
}]
```

---

_[View the full topic](https://discuss.elastic.co/t/duplicate-events-with-filebeat-on-windows-on-service-restart/78743)._