Elastic Indexes and Data Streams

Hello everyone,

I was wondering how should I make my policy for Elastic to clean the space up on a 200GB limited space VM.

I studied deeply the config of Index Policies and I've set my custom ones for my needs.

Since I'm just using the trial of ES 9 (for the Defend feature, which rocks), but in like 5 days (with Windows, Syslog, Defend and Auditd Logs) it gets almost full.

This is my current Index Policy (hoping it makes actually sense):

My point was:
If the index is 1 day older or exceeds 10gb do rollup.
Then push the current to warm and make it read only when it's 2 day older.
FInally delete it when it's 3 days old.

But my actual goal is just to clean up space when we pass like 2days, since I'm just doing a trial and I get a lot of events (all of the, not just Defend one)

Surely my config is wrong and I assumed so, so i forced on the data stream this config:

Effective data retention
5 days // For now 5 just to see if this worked.

Do you guys have any tips or advice for my needs? Let me know if you need any other info!
Probably I just mis-read and mis-understood the docs, so blame it on me :slight_smile:

Hello,

Could you please share what is the count of HOT Nodes, WARM nodes , their disk size? along with this what is the avg daily size of data which is received ?

This will help to provide better suggestion of the ILM policy configuration we should configure to keep the disk space below 85% of 200GB

Thanks!!