# Filebeat does not harvest files after a restart

**URL:** https://discuss.elastic.co/t/filebeat-does-not-harvest-files-after-a-restart/159608
**Category:** Beats
**Tags:** filebeat
**Created:** [December 5, 2018, 9:47pm UTC](https://discuss.elastic.co/t/filebeat-does-not-harvest-files-after-a-restart/159608 "2018-12-05T21:47:51Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Amit\_Periyapatna](https://avatars.discourse-cdn.com/v4/letter/a/f4b2a3/32.png) [@Amit\_Periyapatna](https://discuss.elastic.co/u/Amit_Periyapatna)
#### Post date: [December 5, 2018, 9:47pm UTC](https://discuss.elastic.co/t/filebeat-does-not-harvest-files-after-a-restart/159608/1 "2018-12-05T21:47:51Z")

</div>

I have configured filebeat to read files from a log directory.

Any time I change a configuration in `filebeat.yml` file and restart filebeat, it stops harvesting files.

If I restart after deleting the registry, everything works fine. I am not sure if it is a bug within filebeat that it requires me to delete the registry every time I change something in the configuration.

Use case:  
I'm trying to add a new log directory with a new field in my filebeat configuration as follows:

```auto
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - "/mylogs/something*"
    - "/mylogs/otherthing*"

  ignore_older: 5m
  scan_frequency: 20s
  fields:
    type: other
  fields_under_root: true
  multiline.pattern: '^[[:space:]]+(at|\.{3})\b|^Caused by:|^[[:space:]]{2}\"|^}'
  multiline.negate: false
  multiline.match: after
# This is the configuration that I added to filebeat before restarting it.
- type: log
  enabled: true
  paths:
    - "/mylogs/newtype*"
  ignore_older: 5m
  fields:
    type: newtype
  fields_under_root: true
  multiline.pattern: '^[[:space:]]+(at|\.{3})\b|^Caused by:|^[[:space:]]{2}\"|^}'
  multiline.negate: false
  multiline.match: after

```

Let me know if any more details are required.

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [December 6, 2018, 1:11pm UTC](https://discuss.elastic.co/t/filebeat-does-not-harvest-files-after-a-restart/159608/2 "2018-12-06T13:11:59Z")

</div>

Filebeat remembers the last processed offsets in a file. It does not send already processed contents when the configuration changed.

---

<div class="post-metadata">

### Author: ![Amit\_Periyapatna](https://avatars.discourse-cdn.com/v4/letter/a/f4b2a3/32.png) [@Amit\_Periyapatna](https://discuss.elastic.co/u/Amit_Periyapatna)
#### Post date: [December 6, 2018, 10:32pm UTC](https://discuss.elastic.co/t/filebeat-does-not-harvest-files-after-a-restart/159608/3 "2018-12-06T22:32:53Z")

</div>

Thank you for your reply. I agree that filebeat does that, but as soon as I change anything on my configuration file and restart filebeat, it does not send any data to logstash unless I delete the registry. I am not sure if I am doing something wrong but it has been happening for the last couple of weeks.

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [December 11, 2018, 1:14pm UTC](https://discuss.elastic.co/t/filebeat-does-not-harvest-files-after-a-restart/159608/4 "2018-12-11T13:14:18Z")

</div>

What kind of changes have you done? Can you share the exact config files before and after change?

When filebeat doesn't send it would be helpful to have a look at debug logs from filebeat.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 8, 2019, 1:14pm UTC](https://discuss.elastic.co/t/filebeat-does-not-harvest-files-after-a-restart/159608/5 "2019-01-08T13:14:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
