Curator delete snapshots

Elastic Stack Elasticsearch
1

Hello.
I have a doubt about deleting indices using curator.
I'm deleting them with the simple curl command, and then deleting physically the files on my server because it doesn't delete it.
My question is if deleting it with curator also deletes the physical file on my hard drive.

Thanks.

2

How do you know which files to delete? Indices are not named files since Elasticsearch 5.x. You should never interact with the files in your path.data directly. You should only interact with data (indices, snapshots, etc.) via the API. If you feel as though the indices are not deleted with an API call (and Curator uses the same API call that your curl command uses), that's something worth investigating.

And I'm also a bit confused. Your subject line is talking about deleting snapshots, not indices. If you're worried that an API call is not deleting snapshots, then we would be having a different conversation.

3

Sorry not to explain me clearly.
I mean: I create a filesystem with a folder with a month name, so, every two months I delete the snapshots via API, but the files remain in the disk, so I delete the folder physically.
If I delete the snapshots with curator, it deletes also the files in the disk?

4

An elasticsearch index is comprised of shards, which are comprised of segments. Snapshots are incremental at the segment level. As subsequent snapshots are taken, Elasticsearch compares the list of segments currently stored in the repository to the segments in the index (or indices) being snapshotted, and only copies new segments. Additionally, the new snapshot adds pointers to the existing segments, so that a restore of said snapshot will restore the new segments as well as the older, existing ones.

What this means is that when you delete a single snapshot, unless it deletes all references (or pointers) to all of the segments, they are not deleted. To repeat: segment files are not deleted from the repository until all pointers/references to those segments are also deleted. Only segments which are no longer referenced by a snapshot are deleted.

I believe this is what you are experiencing: You are deleting a snapshot or some snapshots, but not all snapshots referencing the segments you want deleted.

1 Like
5

Oh! It could be, because I mistaken when creating the folder and make all the months be in the same folder, but naming it not the same.
So, I created a backup folder for the snapshots and then, when creating the snapshots, I create them using backup/month/month-day so If I visualize it with elastichq I can see all the snapshots from month 04 and 05. Then they were sharing the same folder...

So basically, I have to delete all the daily snapshots to delete the monthly snapshot isn't it? Deleting only the month snapshot will not delete the files of the disk.

Thank you very much for your explanation!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.