What is an easy way to convert an index to an alias that gets rolled over by ILM?

[jerrac (David Reagan)]

So, during maintenance Friday evening I found that one of my indexes is well over 300-400GB in size and is using up most of my disk space.

I had messed up the configuration of the index/alias I have Filebeat pointed at. The result was the index was not rolling over like it is supposed to.

(Long story, but I have a lot of custom stuff for reasons not relevant to this specific issue other than that they allowed this situation to happen. I'll be reviewing what, exactly, I messed up there tomorrow.)

The solution I wanted to implement was to rename the index so I could use the index name as the alias it is supposed to be, then hopefully things would just work themselves out as ILM kicked in.

Of course, renaming indices is not directly supported. As far as I can tell, the only way to "rename" is to reindex. So, the best I idea I had was to reindex into a temporary index, then reconfigure things to make the old one work right, and then reindex into it again...

It was rather late and my attempts at that didn't really work out. Since as far as I could tell the disk space was not running out super quickly, I put it off until Monday.

I decided to post this today in the interests of maybe getting a head start on tomorrow morning, plus a bit of boredom.

Which leads to my main question: If you have an index that you want to be an alias, how do you convert it to an alias?

Other questions:

I remember something about deleting docs from an index doesn't actually recover disk space until something else happens. Is that still true?

Maybe I didn't read the reindex docs carefully enough, but does reindexing a doc make a copy, or actually move it to the new index and delete it out of the old?

Why isn't renaming an index directly supported? My situation could get annoying fast if I end up doubling how much space the data in the rouge index is using up while reindexing...

[Christian_Dahlqvist]

@jerrac

Which leads to my main question: If you have an index that you want to be an alias, how do you convert it to an alias?

Instead of reindexing the whole thing you could use the clone index API and then delete the original index, which would free up the name for use as an alias. That should be a lot quicker than reindexing.

@jerrac

I remember something about deleting docs from an index doesn't actually recover disk space until something else happens. Is that still true?

Yes. Deletes in Elasticsearch are soft deletes (updates with empty documents) so disk space is only recovered once segments are merged.

@jerrac

Maybe I didn't read the reindex docs carefully enough, but does reindexing a doc make a copy, or actually move it to the new index and delete it out of the old?

It just indexes the data into a new index and leave the old index intact and unchanged.

[jerrac (David Reagan)]

Well, clone did not work. I only have one node with enough space, and ES won't let the clone happen because the result would take the disk space to well over the allowed limit.

My current plan is to reindex by date until it stops letting me, then delete whatever I couldn't reindex.

Not ideal, but some of the older data should have aged out by now anyway.

Thanks for answering my questions about deleting and reindexing.