Automatically deleting logs


(san) #1

When the storage of Elasticsearch is full, i want to get the oldest logs
automatically deleted to accommodate new logs.

Could someone help me in achieving this?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0f335836-3395-4071-9b79-33d3505d6e5a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #2

I think you are looking for this: https://github.com/elasticsearch/elasticsearch/issues/2114

By now, you have to manage it yourself.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 févr. 2014 à 07:35, san sandks14@gmail.com a écrit :

When the storage of Elasticsearch is full, i want to get the oldest logs automatically deleted to accommodate new logs.

Could someone help me in achieving this?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0f335836-3395-4071-9b79-33d3505d6e5a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/E3BC7678-D805-475E-9C1E-E6865F551C30%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.


(san) #3

Hi David,

Thanks for you response.

My requirement is to get the old logs deleted due to non-availability of
STORAGE SPACE to accommodate new logs.
This is regardless of ttl is expired or not.

On Friday, February 28, 2014 12:55:01 PM UTC+5:30, David Pilato wrote:

I think you are looking for this:
https://github.com/elasticsearch/elasticsearch/issues/2114

By now, you have to manage it yourself.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 févr. 2014 à 07:35, san <sand...@gmail.com <javascript:>> a écrit :

When the storage of Elasticsearch is full, i want to get the oldest logs
automatically deleted to accommodate new logs.

Could someone help me in achieving this?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/0f335836-3395-4071-9b79-33d3505d6e5a%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dce0efc5-9bd8-4656-baf6-e317aca28e28%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #4

It does not exist.
I really think that you should run a cron job and decide what to do in that case.

But why not removing logs after a given period?

BTW when I say delete logs I mean remove daily index. Deleting logs entry will cost you more space due to how Lucene works behind the scene.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 févr. 2014 à 09:01, san sandks14@gmail.com a écrit :

Hi David,

Thanks for you response.

My requirement is to get the old logs deleted due to non-availability of STORAGE SPACE to accommodate new logs.
This is regardless of ttl is expired or not.

On Friday, February 28, 2014 12:55:01 PM UTC+5:30, David Pilato wrote:
I think you are looking for this: https://github.com/elasticsearch/elasticsearch/issues/2114

By now, you have to manage it yourself.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 févr. 2014 à 07:35, san sand...@gmail.com a écrit :

When the storage of Elasticsearch is full, i want to get the oldest logs automatically deleted to accommodate new logs.

Could someone help me in achieving this?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0f335836-3395-4071-9b79-33d3505d6e5a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dce0efc5-9bd8-4656-baf6-e317aca28e28%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2D09CA87-E272-4E08-ACB6-CB8DBED65233%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.


(Mark Walkom) #5

Take a look at https://github.com/elasticsearch/curator

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 28 February 2014 19:07, David Pilato david@pilato.fr wrote:

It does not exist.
I really think that you should run a cron job and decide what to do in
that case.

But why not removing logs after a given period?

BTW when I say delete logs I mean remove daily index. Deleting logs entry
will cost you more space due to how Lucene works behind the scene.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 févr. 2014 à 09:01, san sandks14@gmail.com a écrit :

Hi David,

Thanks for you response.

My requirement is to get the old logs deleted due to non-availability of
STORAGE SPACE to accommodate new logs.
This is regardless of ttl is expired or not.

On Friday, February 28, 2014 12:55:01 PM UTC+5:30, David Pilato wrote:

I think you are looking for this: https://github.com/
elasticsearch/elasticsearch/issues/2114

By now, you have to manage it yourself.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 févr. 2014 à 07:35, san sand...@gmail.com a écrit :

When the storage of Elasticsearch is full, i want to get the oldest logs
automatically deleted to accommodate new logs.

Could someone help me in achieving this?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/0f335836-3395-4071-9b79-33d3505d6e5a%
40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dce0efc5-9bd8-4656-baf6-e317aca28e28%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2D09CA87-E272-4E08-ACB6-CB8DBED65233%40pilato.fr
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624aqxU-A1B8DB__GwmdM7hgMFY7iQgwQj37p_0GdgZjhuw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #6