# Delete logs in ElasticSearch after certain period **URL:** \ **Category:** Elasticsearch\ **Created:** [February 14, 2017, 3:22pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067 "2017-02-14T15:22:15Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\ **Post date:** [February 14, 2017, 3:22pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/1 "2017-02-14T15:22:15Z")
I have tons of logs that was writing to elasticsearch service . i was running out of space its keep writing logs . i was looking for something to delete logs after certain period of time. Ex: 15days or 20days or 1mnth automatically . Is there any option or way available in elasticsearch. I was using ELKB. Filebeat-Logstash-ElasticSearch-Kibana. I hope there was something available but i was not sure maybe because of lack of knowledge. ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [February 14, 2017, 3:28pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/2 "2017-02-14T15:28:56Z")
Hi, Are you talking about Elasticsearch log files or log indexed in Elasticsearch and searchable using Kibana? ---
**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\ **Post date:** [February 14, 2017, 3:32pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/3 "2017-02-14T15:32:29Z")
> [@tanguy](#): > > log indexed in Elasticsearch and searchable using Kibana was talking about log indexed in Elasticsearch. I was sending large amount of logs to ES , so want to delete logs(mydata) after 15 or 20 days. ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [February 14, 2017, 3:34pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/4 "2017-02-14T15:34:49Z")
Ok. You can have a look at Curator ([https://www.elastic.co/guide/en/elasticsearch/client/curator/5.0/index.html](https://www.elastic.co/guide/en/elasticsearch/client/curator/5.0/index.html)). ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 14, 2017, 5:09pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/5 "2017-02-14T17:09:06Z")
Curator 5.0 is not yet released. Try [https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) ---
**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\ **Post date:** [February 14, 2017, 5:22pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/6 "2017-02-14T17:22:17Z")
> [@theuntergeek](#): > > Curator 5.0 is not yet released. Try [Curator Reference [8.0] | Elastic](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) I just want to delete my old data what ever i send to Elasticsearch. I was using elasticsearch 5.1. Is there any chance to delete old data ?????? Please direct me proper way to delete those automatically. ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 14, 2017, 5:37pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/7 "2017-02-14T17:37:24Z")
@bob-bza, [Elasticsearch Curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) _is_ the way to automatically delete your older indices. See the [delete\_indices](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/delete_indices.html) action documentation, and the [example](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/ex_delete_indices.html) for the same action. The rest of the documentation should help with understanding. ---
**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\ **Post date:** [February 14, 2017, 5:40pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/8 "2017-02-14T17:40:21Z")
> [@theuntergeek](#): > > @bob-bza, Elasticsearch Curator is the way to automatically delete your older indices. > > See the delete\_indices action documentation, and the example for the same action. The rest of the documentation should help with understanding. I was using Elasticsearch 5.1 in AWS does it support Elasticsearch Curator ??? ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 14, 2017, 5:47pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/9 "2017-02-14T17:47:57Z")
If you are running your _own_ installation of Elasticsearch in your own AWS EC2 instance, then Curator will work for you. _However..._ If you are running AWS ES 5.1, then Curator will not work for you. Even though AWS added the `/_cluster/state` endpoint—which Curator depends on—to their release of AWS ES 5.1, it still doesn't have the necessary data to support Curator. See [https://github.com/elastic/curator/issues/880](https://github.com/elastic/curator/issues/880) for more information. I'm sorry for the inconvenience. You will have to write your own scripts to automate index deletion, or you may be able to find some online, somewhere. ---
**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\ **Post date:** [February 14, 2017, 6:38pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/10 "2017-02-14T18:38:36Z")
> [@theuntergeek](#): > > If you are running AWS ES 5.1, then Curator will not work for you. Even though AWS added the /\_cluster/state endpoint—which Curator depends on—to their release of AWS ES 5.1, it still doesn't have the necessary data to support Curator. See [AWS Elasticsearch version 5.x support? · Issue #880 · elastic/curator · GitHub](https://github.com/elastic/curator/issues/880) for more information. If i use AWS ES service 2.3 , does it supports to delete old data?? If it yes means , in that case i will try to use AWS E.S Service 2.3. ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 14, 2017, 6:52pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/11 "2017-02-14T18:52:33Z")
Curator v4 does not work with _any_ version yet released of AWS ES. Curator v3 _should_ work for you with AWS ES 2.x, but lacks many advanced features found in Curator 4. Curator v3 is deprecated and no longer supported in any way. ---
**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\ **Post date:** [February 14, 2017, 7:25pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/12 "2017-02-14T19:25:53Z")
> [@theuntergeek](#): > > Curator v4 does not work with any version yet released of AWS ES. Curator v3 should work for you with AWS ES 2.x, but lacks many advanced features found in Curator 4. Curator v3 is deprecated and no longer supported in any way. So to be clear and straight, AWS ES using any version , i cant delete my old data?? If there is by anychance to delete old data means Please let me know. ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 14, 2017, 8:09pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/13 "2017-02-14T20:09:08Z")
You can always use `curl -XDELETE http://localhost:9200/INDEXNAME` to delete indices. You just won't be able to use Elasticsearch Curator to help automate the process. ---
**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\ **Post date:** [February 14, 2017, 8:29pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/14 "2017-02-14T20:29:45Z")
> [@theuntergeek](#): > > You can always use curl -XDELETE [http://localhost:9200/INDEXNAME](http://localhost:9200/INDEXNAME) to delete indices. You just won't be able to use Elasticsearch Curator to help automate the process. If i run this this will delete all data and index. i want to run something in corn or in settings to delete logs automatically every 15days . ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 14, 2017, 8:39pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/15 "2017-02-14T20:39:58Z")
> [@bob-bza](#): > > If i run this this will delete all data and index. No, that `curl` command will _only_ delete the specified `INDEXNAME`. > [@bob-bza](#): > > i want to run something in corn or in settings to delete logs automatically every 15days . You're on your own for that, as I stated. Curator is normally the go-to solution for that, but AWS made their version of ES incompatible with it. You will have to run the above command manually, or find some way to script it. ---
**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\ **Post date:** [February 14, 2017, 8:46pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/16 "2017-02-14T20:46:42Z")
> [@theuntergeek](#): > > No, that curl command will only delete the specified INDEXNAME. I already used this , it was delete whole log in that index. curl -XDELETE '[http://search-mydomain/s3-logs](http://search-mydomain/s3-logs)' this will delete everything in this index right, i always want to keep for 15days logs. > [@theuntergeek](#): > > You're on your own for that, as I stated. Curator is normally the go-to solution for that, but AWS made their version of ES incompatible with it. You will have to run the above command manually, or find some way to script it. you said AWS was not supporting Curator , so can i use this DELETE /index/type/\_query { "query": { "range": { "@timestamp": { "lt": "now-7d" } } } } To delete my old logs in elasticsearch from logstash. ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 14, 2017, 8:54pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/17 "2017-02-14T20:54:53Z")
It's apparent you are not using time-series indices, if that is the case. You should not be feeding a constant stream of data to a single index unless you're planning on using the rollover API. I already responded to your other request about the delete-by-query, which is a really bad approach to data management as it heavily taxes the cluster making millions of atomic flag-for-delete operations, which then have to be singled out for deletion at the next segment merge operation. In short, do not use this approach if you want your cluster to behave in a performant way. ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 14, 2017, 9:00pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/18 "2017-02-14T21:00:25Z")
I highly recommend looking into the Rollover API for a way to simplify this for you. Then you can make your "non-time-series" index into a time-series index for all intents and purposes. And ask for help with the Rollover API in a new topic, or search for an existing one, as this is off-topic here. ---
**Author:** ![bob-bza](https://avatars.discourse-cdn.com/v4/letter/b/9f8e36/32.png) [@bob-bza](https://discuss.elastic.co/u/bob-bza)\ **Post date:** [February 16, 2017, 8:14pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/20 "2017-02-16T20:14:46Z")
> [@theuntergeek](#): > > I highly recommend looking into the Rollover API for a way to simplify this for you. Then you can make your "non-time-series" index into a time-series index for all intents and purposes. > > And ask for help with the Rollover API in a new topic, or search for an existing one, as this is off-topic here. can i install Curator on my cluster instance and from there can do this > **[delete\_indices | Curator Reference \[8.0\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/ex_delete_indices.html)** in that case does it works on ElasticSearchService in AWS. ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [February 16, 2017, 9:33pm UTC](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067/21 "2017-02-16T21:33:52Z")
> [@bob-bza](#): > > in that case does it works on ElasticSearchService in AWS. No, it doesn't. [Next page](https://discuss.elastic.co/t/delete-logs-in-elasticsearch-after-certain-period/75067.md?page=2)