Deleting Elasticsearch indices without breaking Kibana visualizations

Elastic Stack Kibana
1

Hello,

How do you prevent Kibana visualizations from breaking due to deleted indices (as described in this thread)? I deleted some of my indices in Elasticsearch and afterwards saw the missing index pattern error in each visualization in my Kibana dashboard. I found the easiest way to resolve the errors was to export and import the visualizations, but I am hoping there is a proper way to delete the indices.

Thanks.

2

One thing you might consider if you aren't doing this already is using index aliases and building your visualizations & index patterns based off of those.

An index alias will let you point to one or more underlying "concrete" indices, and you can update an alias at any time to add/remove the backing indices it represents.

Assuming the indices backing your alias aren't removing the fields your visualizations rely on, you should be able to delete them behind the scenes without visualizations knowing anything has changed.

3

Thanks for responding. I'll try using an index alias, but I didn't delete the index pattern, just a subset of the indices that followed the pattern, so I'm not clear on why the particular error (i.e., Could not locate that index-pattern (id: ...), click here to re-create it) I mentioned displayed on all of my visualizations. This sounds like a bug because I would expect the data associated with the remaining indices to display for the visualizations.

4

Ah interesting -- then yes in that case I would expect it to still work.

After testing this a bit, I'm having trouble reproducing the issue you describe. Here's what I'm doing; let me know if it lines up with what you did:

PUT /logs-a

PUT /logs-b

POST /logs-a/_doc
{
  "hello": "world"
}

POST /logs-b/_doc
{
  "hello": "universe"
}

PUT /logs-a/_alias/c

PUT /logs-b/_alias/c

So now we have indices logs-a and logs-b. I also created an alias c which points to these indices for testing purposes.

Then I created two index patterns: logs-* and c.

Next I created and saved a visualization & dashboard for each index pattern (simple datatable doing a terms aggregation on hello.keyword).

Then I deleted logs-b:

DELETE /logs-b

Afterwards, I opened all of the dashboards & visualizations, and everything worked. Refreshed the fields in both index patterns; everything still worked.

What am I missing to make thie error happen?

In the thread you point to about visualizations breaking due to deleted indices, the OP wasn't experiencing problems because the indices were deleted per se, but rather because a new index pattern was created later (with a new ID).

Assuming you don't delete & recreate an index pattern, I would expect everything to continue working, even when an index is deleted.

5

The main difference, I am guessing, is that I didn't manually create the indices, they were automatically created in Elasticsearch (in AWS.) In Kibana, under Dev Tools, I just deleted a a few of the indices using:

DELETE /index-name

And when I loaded the dashboard, that's when I saw the error in each visualization.

I added the index alias, but I will have to wait until next week to attempt another index deletion and will let you know. Do I need to set the Default index (defaultIndex) to the alias' ID, as well?

Thanks.

6

No you shouldn't -- in my test cases I had a different default index and everything still worked.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.