Automatically deleting logs

Elastic Stack Elasticsearch
1

When the storage of Elasticsearch is full, i want to get the oldest logs
automatically deleted to accommodate new logs.

Could someone help me in achieving this?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0f335836-3395-4071-9b79-33d3505d6e5a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

I think you are looking for this: https://github.com/elasticsearch/elasticsearch/issues/2114

By now, you have to manage it yourself.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 févr. 2014 à 07:35, san sandks14@gmail.com a écrit :

When the storage of Elasticsearch is full, i want to get the oldest logs automatically deleted to accommodate new logs.

Could someone help me in achieving this?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0f335836-3395-4071-9b79-33d3505d6e5a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/E3BC7678-D805-475E-9C1E-E6865F551C30%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi David,

Thanks for you response.

My requirement is to get the old logs deleted due to non-availability of
STORAGE SPACE to accommodate new logs.
This is regardless of ttl is expired or not.

On Friday, February 28, 2014 12:55:01 PM UTC+5:30, David Pilato wrote:

I think you are looking for this:
Allow setting ttl at index level · Issue #2114 · elastic/elasticsearch · GitHub

By now, you have to manage it yourself.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 févr. 2014 à 07:35, san <sand...@gmail.com <javascript:>> a écrit :

When the storage of Elasticsearch is full, i want to get the oldest logs
automatically deleted to accommodate new logs.

Could someone help me in achieving this?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/0f335836-3395-4071-9b79-33d3505d6e5a%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dce0efc5-9bd8-4656-baf6-e317aca28e28%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

It does not exist.
I really think that you should run a cron job and decide what to do in that case.

But why not removing logs after a given period?

BTW when I say delete logs I mean remove daily index. Deleting logs entry will cost you more space due to how Lucene works behind the scene.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 févr. 2014 à 09:01, san sandks14@gmail.com a écrit :

Hi David,

Thanks for you response.

My requirement is to get the old logs deleted due to non-availability of STORAGE SPACE to accommodate new logs.
This is regardless of ttl is expired or not.

On Friday, February 28, 2014 12:55:01 PM UTC+5:30, David Pilato wrote:
I think you are looking for this: Allow setting ttl at index level · Issue #2114 · elastic/elasticsearch · GitHub

By now, you have to manage it yourself.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 févr. 2014 à 07:35, san sand...@gmail.com a écrit :

When the storage of Elasticsearch is full, i want to get the oldest logs automatically deleted to accommodate new logs.

Could someone help me in achieving this?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0f335836-3395-4071-9b79-33d3505d6e5a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dce0efc5-9bd8-4656-baf6-e317aca28e28%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2D09CA87-E272-4E08-ACB6-CB8DBED65233%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.

5

Take a look at GitHub - elastic/curator: Curator: Tending your Elasticsearch indices

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 28 February 2014 19:07, David Pilato david@pilato.fr wrote:

It does not exist.
I really think that you should run a cron job and decide what to do in
that case.

But why not removing logs after a given period?

BTW when I say delete logs I mean remove daily index. Deleting logs entry
will cost you more space due to how Lucene works behind the scene.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 févr. 2014 à 09:01, san sandks14@gmail.com a écrit :

Hi David,

Thanks for you response.

My requirement is to get the old logs deleted due to non-availability of
STORAGE SPACE to accommodate new logs.
This is regardless of ttl is expired or not.

On Friday, February 28, 2014 12:55:01 PM UTC+5:30, David Pilato wrote:

I think you are looking for this: https://github.com/
elasticsearch/elasticsearch/issues/2114

By now, you have to manage it yourself.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 févr. 2014 à 07:35, san sand...@gmail.com a écrit :

When the storage of Elasticsearch is full, i want to get the oldest logs
automatically deleted to accommodate new logs.

Could someone help me in achieving this?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/0f335836-3395-4071-9b79-33d3505d6e5a%
40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dce0efc5-9bd8-4656-baf6-e317aca28e28%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2D09CA87-E272-4E08-ACB6-CB8DBED65233%40pilato.fr
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624aqxU-A1B8DB__GwmdM7hgMFY7iQgwQj37p_0GdgZjhuw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.