# Filebeats(5.40) keeps on recreating the indices after deletion

**URL:** <https://discuss.elastic.co/t/filebeats-5-40-keeps-on-recreating-the-indices-after-deletion/121986>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 1, 2018, 6:16am UTC](https://discuss.elastic.co/t/filebeats-5-40-keeps-on-recreating-the-indices-after-deletion/121986 "2018-03-01T06:16:03Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![dubul](https://avatars.discourse-cdn.com/v4/letter/d/9f8e36/32.png) [@dubul](https://discuss.elastic.co/u/dubul)\
**Post date:** [March 1, 2018, 6:16am UTC](https://discuss.elastic.co/t/filebeats-5-40-keeps-on-recreating-the-indices-after-deletion/121986/1 "2018-03-01T06:16:03Z")

</div>

I have filebeats reading the log files on a remote server and shipping it to logstash on the same server. I have tried deleting the last 6 months old indices but I have noticed that indices are recreated on the elastic search. Can you correct me where i'm doing wrong I went with the default configuration. Thanks

```
  paths:
    #- /var/log/*.log
    - \\webserver10\iislogs\*.log

output.logstash:
  # Boolean flag to enable or disable the output module.
  #enabled: true

  # The Logstash hosts
  hosts: ["localhost:5044"]

  # Number of workers per Logstash host.
  #worker: 1
```

---

<div class="post-metadata">

**Author:** ![tag\_v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tag_v/32/47466_2.png) [@tag\_v](https://discuss.elastic.co/u/tag_v)\
**Post date:** [March 1, 2018, 11:41am UTC](https://discuss.elastic.co/t/filebeats-5-40-keeps-on-recreating-the-indices-after-deletion/121986/2 "2018-03-01T11:41:13Z")

</div>

filebeat is only a log shipper. it wont handle index creations. check logstash conf if logstash output elasticsearch contains that index names.

---

<div class="post-metadata">

**Author:** ![dubul](https://avatars.discourse-cdn.com/v4/letter/d/9f8e36/32.png) [@dubul](https://discuss.elastic.co/u/dubul)\
**Post date:** [March 5, 2018, 6:30pm UTC](https://discuss.elastic.co/t/filebeats-5-40-keeps-on-recreating-the-indices-after-deletion/121986/3 "2018-03-05T18:30:58Z")

</div>

Here is the output config for logstash.

```
output {
	elasticsearch {
		"hosts" => ["elasticsearch:80"]
		"index" => "iis_logs-%{+YYYY.MM}"
		"document_type" => "iislog"
	}	
}
```

---

<div class="post-metadata">

**Author:** ![tag\_v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tag_v/32/47466_2.png) [@tag\_v](https://discuss.elastic.co/u/tag_v)\
**Post date:** [March 5, 2018, 8:52pm UTC](https://discuss.elastic.co/t/filebeats-5-40-keeps-on-recreating-the-indices-after-deletion/121986/4 "2018-03-05T20:52:02Z")

</div>

seems filebeat still reading logs from the path sending to logstash. and logstash reading logs and pushing them to index iis\_logs-\* . make sure no filebeat is sending logs to logstash.

---

<div class="post-metadata">

**Author:** ![dubul](https://avatars.discourse-cdn.com/v4/letter/d/9f8e36/32.png) [@dubul](https://discuss.elastic.co/u/dubul)\
**Post date:** [March 6, 2018, 12:16am UTC](https://discuss.elastic.co/t/filebeats-5-40-keeps-on-recreating-the-indices-after-deletion/121986/5 "2018-03-06T00:16:40Z")

</div>

I notice from the filebeats registry that filebeats is re-reading the old untouched log files and shipping it to logstash. Is there any config setting which stop filebeats in re-reading the logs or stop shipping old log files to logstash?

---

<div class="post-metadata">

**Author:** ![tag\_v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tag_v/32/47466_2.png) [@tag\_v](https://discuss.elastic.co/u/tag_v)\
**Post date:** [March 6, 2018, 5:24am UTC](https://discuss.elastic.co/t/filebeats-5-40-keeps-on-recreating-the-indices-after-deletion/121986/6 "2018-03-06T05:24:25Z")

</div>

`ignore_older: 3h`  
This you can use to ignore older files. Moreover you can change your registry file if you are sure filebeat pushing logs from old registry with:

```
registry_file: 'C:\ProgramData\Filebeat\registry' (if windows)
registry_file: '/var/registrydata/filebeat/registry' (if linux)
```

---

<div class="post-metadata">

**Author:** ![dubul](https://avatars.discourse-cdn.com/v4/letter/d/9f8e36/32.png) [@dubul](https://discuss.elastic.co/u/dubul)\
**Post date:** [March 6, 2018, 5:41am UTC](https://discuss.elastic.co/t/filebeats-5-40-keeps-on-recreating-the-indices-after-deletion/121986/7 "2018-03-06T05:41:30Z")

</div>

Sorry what do you want me to change in the registry file in windows?  
Thanks for your response.

---

<div class="post-metadata">

**Author:** ![tag\_v](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tag_v/32/47466_2.png) [@tag\_v](https://discuss.elastic.co/u/tag_v)\
**Post date:** [March 6, 2018, 5:48am UTC](https://discuss.elastic.co/t/filebeats-5-40-keeps-on-recreating-the-indices-after-deletion/121986/8 "2018-03-06T05:48:51Z")

</div>

Ya try changing registry file path with your respective OS. and use `ignore_older` as well.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 3, 2018, 5:49am UTC](https://discuss.elastic.co/t/filebeats-5-40-keeps-on-recreating-the-indices-after-deletion/121986/9 "2018-04-03T05:49:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.