# After deleting an index, a new one is created with invalid naming scheme **URL:** \ **Category:** Logstash\ **Created:** [December 22, 2021, 11:48pm UTC](https://discuss.elastic.co/t/after-deleting-an-index-a-new-one-is-created-with-invalid-naming-scheme/292747 "2021-12-22T23:48:23Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![mdebord](https://avatars.discourse-cdn.com/v4/letter/m/e9c0ed/32.png) [@mdebord](https://discuss.elastic.co/u/mdebord)\ **Post date:** [December 22, 2021, 11:48pm UTC](https://discuss.elastic.co/t/after-deleting-an-index-a-new-one-is-created-with-invalid-naming-scheme/292747/1 "2021-12-22T23:48:23Z")
> Logstash 7.16.2 > Elasticsearch 7.16.2 # **Summary** In a scenario that beat processors are sending to logstash, and logstash has ilm configured, if the working index is deleted, a new one is created (as expected), but the naming convention is wrong. I realize there should not be a reason for an index to be deleted in this way, but in the event that it did happen, it would be expected that the system would create a new index with the appropriate name. Things like index patterns/etc can break when indexes are not formatted as expected. # **Problem Scenario:** Given this logstash config: ```auto input { beats { port => 5000 } } output { elasticsearch { hosts => ["http://localhost:9200"] ilm_enabled => true ilm_rollover_alias => "filebeat" ilm_pattern => "{now/d}-000001" ilm_policy => "30-days-default" } } ``` When the beats publisher starts to push into logstash, the index name format is automatically created as expected (ex: `filebeat-2021.12.20-000001`) With the above scenario prepared, **steps to reproduce** : 1. Delete the current index that was created (ex: `filebeat-2021.12.20-000001`) 2. Wait for the beats publisher to push more messages to logstash 3. A new index is automatically created **Expected behavior:** - The new index that was created should follow the same naming scheme (ex: `filebeat-2021.12.20-000001`, `filebeat-2021.12.20-000002`, etc) **Actual behavior:** - The new index created is only the rollover alias: (ex: `filebeat`) with no pattern applied ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [December 23, 2021, 12:08am UTC](https://discuss.elastic.co/t/after-deleting-an-index-a-new-one-is-created-with-invalid-naming-scheme/292747/2 "2021-12-23T00:08:50Z")
Yes that is the expected behavior because once you delete the actual index there is no longer an index that has the correct write alias. When you deleted the index, you deleted the alias as well. It's not best practice to just delete a managed index. What happens is then filebeat / logstash just starts writing to the alias as if it's the correct actual index because there is no alias to point it to the new index. The rollover needs to happen first! You either need to A) Force the rollover before you delete index. This is a good approach because if you just deleted and you have multiple beats writing, it'll immediately write and you will see the behavior that you see above. If you force to roll over first then the data is being written to the new induction. You can delete the old one. `POST filebeat/_rollover` > **[Rollover API | Elasticsearch Guide \[7.16\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-rollover-index.html#indices-rollover-index)** Otherwise You will have to stop all instances of logstash / filebeat Or Manually create the index with the alias known as the initial managed index > **[Configure a lifecycle policy | Elasticsearch Guide \[7.16\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/set-up-lifecycle-policy.html#create-initial-index)** There is also a question of how / If you initially ran filebeat setup, That is what loaded the templates and created the initial managed index. So you can also run filebeat set up again... **Corrected** : Logstash does create that initial managed index as well And of course, if you didn't want to set all that in logstash, you can just use basically a pass-through output let filebeat manage it. It would look something like this ```auto ################################################ # beats->logstash->es default config. ################################################ input { beats { port => 5044 } } output { if [@metadata][pipeline] { elasticsearch { hosts => "http://localhost:9200" manage_template => false index => "%{[@metadata][beat]}-%{[@metadata][version]}" pipeline => "%{[@metadata][pipeline]}" user => "elastic" password => "secret" } } else { elasticsearch { hosts => "http://localhost:9200" manage_template => false index => "%{[@metadata][beat]}-%{[@metadata][version]}" user => "elastic" password => "secret" } } } ``` ---
**Author:** ![mdebord](https://avatars.discourse-cdn.com/v4/letter/m/e9c0ed/32.png) [@mdebord](https://discuss.elastic.co/u/mdebord)\ **Post date:** [December 23, 2021, 2:36am UTC](https://discuss.elastic.co/t/after-deleting-an-index-a-new-one-is-created-with-invalid-naming-scheme/292747/3 "2021-12-23T02:36:44Z")
**Thanks so much for the response, that seems to clear it up, but I have a few follow ups based on your reply:** > [@stephenb](#): > > Logstash does not create that initial managed index. In my stack, filebeat has no access to elasticsearch directly, and the filebeat auto setup `filebeat setup` was never ran, everything was done manually. The first index was created automatically when logstash pushed the first message (and no index existed) and it followed the correct scheme and attached the correct lifecycle policy. When I restart the logstash service, the index is recreated in proper format - so it must be logstash, right? I don't know what else would have created the index. Lifecycles and everything seem to be working fine with the auto created index. In other words, if I stop logstash, delete the "bad" index, and restart logstash: a new index is created with the correct naming convention. What could have done that other than logstash? This is actually the initial reason I thought this behavior of the wrong index name was a bug. > [@stephenb](#): > > There is also a question of how / If you initially ran filebeat setup, That is what loaded the templates and created the initial managed index. Manual setup. Imported the template and used one of the existing lifecycle policies. Everything seems to be working fine regarding lifecycles and the index picking up the template schema. > [@stephenb](#): > > It's not best practice to just delete a managed index For sure, I was thinking in more of a DR scenario that the index for some reason was deleted, corrupted, etc. Just testing how the system would respond in that scenario. > [@stephenb](#): > > POST filebeat-2021.12.20-000001/\_rollover That didn't work, but hitting the alias did: `POST filebeat/_rollover` ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [December 23, 2021, 3:11am UTC](https://discuss.elastic.co/t/after-deleting-an-index-a-new-one-is-created-with-invalid-naming-scheme/292747/4 "2021-12-23T03:11:31Z")
> [@mdebord](#): > > When I restart the logstash service, the index is recreated in proper format - so it must be logstash, right? Yup makes sense, yes it appears logstash does it as well good to know, I didn't have my lab to test it but now we know. **Corrected** above > [@mdebord](#): > > That didn't work, but hitting the alias did: `POST filebeat/_rollover` Yup typo on my end... Fixed Glad you got it working.. Bottom line, don't delete that index while it's running. Roll it over first or create a new bootstrap index with the write alias. ---
**Author:** ![mdebord](https://avatars.discourse-cdn.com/v4/letter/m/e9c0ed/32.png) [@mdebord](https://discuss.elastic.co/u/mdebord)\ **Post date:** [December 28, 2021, 5:32pm UTC](https://discuss.elastic.co/t/after-deleting-an-index-a-new-one-is-created-with-invalid-naming-scheme/292747/5 "2021-12-28T17:32:28Z")
Why is this solved? I never marked a solution because the original report was not addressed, yet it's marked as solved. I just spend the first few posts explaining how it's actually Logstash creating and naming the index, then the post is marked as solved? It's realized that you're not supposed to just go around deleting indexes like that, but DR events are not "expected" nor "best practice" events. Even unexpected or not best practice, it's expected that software systems to behave in a reasonable way. _ **It's reasonable to assume that Logstash would create a new index using the same naming scheme that it used when it first created the index.** _ If this could be fixed in Logstash, it would eliminate a spiderweb of problems in the "unexpected" and "unplanned" even that an index was deleted before rotating. It's fine if this is a _"will not fix"_ judgement by Logstash team, but it's definitely not a "solved" issue. One would reasonably expect whatever process creates index to be responsible for naming them properly even in the event that an index needs to be unexpectedly recreated. ---
**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\ **Post date:** [December 28, 2021, 7:09pm UTC](https://discuss.elastic.co/t/after-deleting-an-index-a-new-one-is-created-with-invalid-naming-scheme/292747/6 "2021-12-28T19:09:15Z")
> [@mdebord](#): > > Why is this solved? I never marked a solution because the original report was not addressed, yet it's marked as solved. This is a community discussion forum, and you left a reply indicating that the original problem was understood so a mod marked it as a solution: > [@mdebord](#): > > Thanks so much for the response, that seems to clear it up If you have an _issue_ that needs escalation to the develpers managing a project, opening an issue on [the relevant project's issue tracker](https://github.com/logstash-plugins/logstash-output-elasticsearch/issues) is a way to track the issue through to resolution. * * * That said, The Elasticsearch output plugin for Logstash doesn't have a way of detecting an Elasticsearch index being deleted. Manual manipulation of policy-managed assets, _especially when those assets are in use_, is a recipe for edge-cases, and as a Logstash dev with knolwedge of the APIs available, I don't see a way to prevent this issue from occurring. When a Logstash pipeline starts, and it has an Elasticsearch output configured with ILM, it ensures that the ILM policy provided is bootstrapped, creating a write alias as appropriate. After it has done so, it pushes events to the write index using the BULK API. As the pipeline processes events, it expects the alias to be in place (because it was there _before_ we started pushing events). When Elasticsearch receives a directive to write to an index that does not exist, it helpfully creates the index (unless policy dictates that it should reject the request). Because Logstash has no way of knowing that the write alias has been deleted, in-flight events that were intended to be routed to the write index are interpreted _by Elasticsearch_ to be routed to an index with the same name, creating the index. If you need to manipulate policy-managed assets manually, you need to first make sure that _all_ activity is shut down first (including pipelines writing to the ILM indices). ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [December 28, 2021, 9:21pm UTC](https://discuss.elastic.co/t/after-deleting-an-index-a-new-one-is-created-with-invalid-naming-scheme/292747/7 "2021-12-28T21:21:39Z")
Apologies ... Removed solved. Solved on the community forum typically means the question is answered / understood it does not necessarily mean the software itself has been changed, etc. to solve the issue. But I understand your perspective. I misunderstood your response as understanding that it works as designed today. Agree with @yaauie above that if you would like to see additional features, please feel free to file a feature request. To me, this behavior is very much like when you have a soft link to a file in Linux to an actual concrete file. If you remove the soft link and then continue to write, it will create a new hard / concrete file with the name of the soft link. That's pretty much the same behavior In this case. You have an alias pointing to a concrete index and when the alias is deleted, a concrete index with that alias name is created. Please we encourage you to open feature request as we highly value our community input. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 25, 2022, 9:22pm UTC](https://discuss.elastic.co/t/after-deleting-an-index-a-new-one-is-created-with-invalid-naming-scheme/292747/8 "2022-01-25T21:22:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.