# Filebeat duplicates events after restart

**URL:** <https://discuss.elastic.co/t/filebeat-duplicates-events-after-restart/293917>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 10, 2022, 5:07pm UTC](https://discuss.elastic.co/t/filebeat-duplicates-events-after-restart/293917 "2022-01-10T17:07:51Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![pilincom](https://avatars.discourse-cdn.com/v4/letter/p/eada6e/32.png) [@pilincom](https://discuss.elastic.co/u/pilincom)\
**Post date:** [January 10, 2022, 5:07pm UTC](https://discuss.elastic.co/t/filebeat-duplicates-events-after-restart/293917/1 "2022-01-10T17:07:51Z")

</div>

I'm using Filebeat 7.16.2 to ship logs to Logstash -\> ES.

For some reason every time I restart filebeat service, it scans all files in the log directory again and sends them to logstash. registry directory is accessible and writable and I can see it being updated.

Here is the config files:

/etc/filebeat/filebeat.yml

```auto
---
name: testapp.company.com
tags: []
fields: {}
fields_under_root: false
filebeat:
  config.inputs:
    enabled: true
    path: "/etc/filebeat/conf.d/*.yml"
  config.modules:
    enabled: false
    path: "/etc/filebeat/modules.d/*.yml"
  shutdown_timeout: '10s'
  modules: []
  registry:
    path: "/var/lib/filebeat"
    file_permissions: '0600'
    flush: 0s
http: {}
cloud: {}
output:
  logstash:
    hosts:
    - logstash.example.com:9008
shipper: {}
logging:
  level: info
  to_files: true
  files:
    path: "/var/log/filebeat"
    name: filebeat
    keepfiles: 7
    permissions: 416
autodiscover: {}
runoptions: {}
processors: []
setup: {}

```

/etc/filebeat/conf.d/app.yml

```auto
---
- type: filestream
  paths:
    - /var/log/app/2022/*/*.log
  encoding: plain
  exclude_lines:
    - 'DEBUG'
    - 'INFO'
    - 'NOTICE'
  parsers:
    - multiline:
        type: pattern
        pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
        negate: true
        match: after
        max_lines: 150
        timeout: '10s'
  fields:
    type: app
    env: dev

```

Registry dir:

```auto
ls -la /var/lib/filebeat/
total 16
drwxrwxrwx 3 root root 4096 Jan 10 16:42 .
drwxr-xr-x 51 root root 4096 Jul 25 2019 ..
drwxr-x--- 2 root root 4096 Jan 10 15:27 filebeat
-rw------- 1 root root 0 Jan 10 16:42 filebeat.lock
-rw------- 1 root root 95 Jan 10 15:27 meta.json

ls -la /var/lib/filebeat/filebeat
total 148
drwxr-x--- 2 root root 4096 Jan 10 15:27 .
drwxrwxrwx 3 root root 4096 Jan 10 16:42 ..
-rw------- 1 root root 131683 Jan 10 16:47 log.json
-rw------- 1 root root 15 Jan 10 15:27 meta.json

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 7, 2022, 7:08pm UTC](https://discuss.elastic.co/t/filebeat-duplicates-events-after-restart/293917/2 "2022-02-07T19:08:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.