# Logstash 5.4 recreating the indices after deletion

**URL:** <https://discuss.elastic.co/t/logstash-5-4-recreating-the-indices-after-deletion/122583>\
**Category:** Logstash\
**Created:** [March 5, 2018, 7:35pm UTC](https://discuss.elastic.co/t/logstash-5-4-recreating-the-indices-after-deletion/122583 "2018-03-05T19:35:49Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![dubul](https://avatars.discourse-cdn.com/v4/letter/d/9f8e36/32.png) [@dubul](https://discuss.elastic.co/u/dubul)\
**Post date:** [March 5, 2018, 7:35pm UTC](https://discuss.elastic.co/t/logstash-5-4-recreating-the-indices-after-deletion/122583/1 "2018-03-05T19:35:49Z")

</div>

I have filebeats reading the log files on a remote server and shipping it to logstash on the same server. I have tried deleting the last 6 months old indices but I have noticed that indices are recreated on the elastic search.

Here is the output configuration:

```
output {
	elasticsearch {
		"hosts" => ["elasticsearch:80"]
		"index" => "iis_logs-%{+YYYY.MM}"
		"document_type" => "iislog"
	}	
}
```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 5, 2018, 8:01pm UTC](https://discuss.elastic.co/t/logstash-5-4-recreating-the-indices-after-deletion/122583/2 "2018-03-05T20:01:30Z")

</div>

If Logstash is recreating the indexes it's because something is sending them to Logstash.

It sounds like Filebeat is re-reading the files, something it normally won't do. When does this happen? Have you looked into the Filebeat registry file which records the file path, inode, and current position of input log files? Are the old log files untouched?

---

<div class="post-metadata">

**Author:** ![dubul](https://avatars.discourse-cdn.com/v4/letter/d/9f8e36/32.png) [@dubul](https://discuss.elastic.co/u/dubul)\
**Post date:** [March 6, 2018, 12:11am UTC](https://discuss.elastic.co/t/logstash-5-4-recreating-the-indices-after-deletion/122583/3 "2018-03-06T00:11:13Z")

</div>

Yes old logs are untouched. But I still notice from the filebeats registry that filebeats is re-reading the old untouched files.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 6, 2018, 6:51am UTC](https://discuss.elastic.co/t/logstash-5-4-recreating-the-indices-after-deletion/122583/4 "2018-03-06T06:51:53Z")

</div>

Well, this is hard to debug remotely. You'll have to keep digging. How do you know the old logs are untouched? Have you increased the Filebeat log level to get additional clues? What goes on in the registry file when it rereads the files?

I suggest you move this topic to the Filebeat category where the Filebeat experts hang out.

---

<div class="post-metadata">

**Author:** ![dubul](https://avatars.discourse-cdn.com/v4/letter/d/9f8e36/32.png) [@dubul](https://discuss.elastic.co/u/dubul)\
**Post date:** [March 6, 2018, 8:38pm UTC](https://discuss.elastic.co/t/logstash-5-4-recreating-the-indices-after-deletion/122583/5 "2018-03-06T20:38:55Z")

</div>

updating filebeat config with the correct registry path fixed the issue.  
`filebeat.registry_file: 'C:\ProgramData\Filebeat\registry'`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 3, 2018, 8:39pm UTC](https://discuss.elastic.co/t/logstash-5-4-recreating-the-indices-after-deletion/122583/6 "2018-04-03T20:39:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.