Automatically Purging Indexes and Backups

Elastic Stack Elasticsearch
1

Hi,
I am new to elastic and am looking to use elastic at the moment for logging and APM. I love it.

Having said that, since we are just getting things up and running, we are limited to one windows production server which I believe is only capable of having one elastic node.

One challenge with just having one node however, is the hot/warm/cold architecture and methodology of retiring data isn't really viable for us. Having said that, I have concerns that our storage space on our server will just continue to grow and grow as we get more indexes every day.

Additionally I have daily snapshots running that also take up more and more storage space.

My question for you guys, is...

  1. Is there a way to back up and delete indexes older than a specific date? Or is there a way to make the single elastic node not use indexes (like freezing them) after a specific date?
  2. Is there a way to delete snapshots after a specific date. Like a month old?

Thanks!
-Paul

2

I think what you are searching for are ILM and SLM. Please note that "hot", "cold" and "warm" are just phases. They don't have to correspond with moving indices from "hot" nodes to "cold" notes. You can assign different actions to these phases or just ignore them having only "delete" phase.

3

Hi Igor,
Are you saying that it's possible to have hot, warm and cold indexes with only one node?

4

Of course. It's the same thing, you just don't reallocate the indices between nodes. You still can still create snapshots of these indices, freeze them and delete them all on a single node. That's how my home cluster is setup.

1 Like
5

Hi Igor, Thank you for responding.

When I set up hot, warm, cold indicies in ILM, I get the following error...

illegal_argument_exception: index.lifecycle.rollover_alias [xxxx-alias] does not point to index [xxxx-2019.11.12]

I was under the impression this was because it was because I am working with only one node. If this isn't the case that's great, but how do I resolve this issue? I've googled this and it seems like a lot of people have reported this issue, but I've seen no real solutions.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.