Controlling Stored Data Size

Elastic Stack Elasticsearch
1

Greetings!
I plain to use elasticache with logstash.
Our production servers send about 20G of logs per day, so indexing server
may run out of disk space very soon.
On my current system (which I want to replace) I have a control over
indexed data rolling mechanism, i.e. I can configure it to archive the old
data and after a some period of time remove it completely from index DB.
Tried to search for an alternative configuration in elasticsearch, but with
no luck.
Can you please advise on what my options are ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hey,

right now there is no expire on indices, but this might come sooner or
later, see Allow setting ttl at index level · Issue #2114 · elastic/elasticsearch · GitHub

Apart from that you can use time based indices (per month), combine them
for search via aliases, and maybe delete them manually via a cronjob at the
beginning of each month, or even have a small monitoring script, which
checks the available space and then deletes the oldest data automatically
for now.

Hope this helps...

--Alex

On Wed, Aug 28, 2013 at 4:35 PM, Michael Leikind <
michael.leikind@cloudon.com> wrote:

Greetings!
I plain to use elasticache with logstash.
Our production servers send about 20G of logs per day, so indexing server
may run out of disk space very soon.
On my current system (which I want to replace) I have a control over
indexed data rolling mechanism, i.e. I can configure it to archive the old
data and after a some period of time remove it completely from index DB.
Tried to search for an alternative configuration in elasticsearch, but
with no luck.
Can you please advise on what my options are ?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.