# Delete old logs

**URL:** <https://discuss.elastic.co/t/delete-old-logs/245268>\
**Category:** Elasticsearch\
**Tags:** ilm-index-lifecycle-management\
**Created:** [August 17, 2020, 3:39pm UTC](https://discuss.elastic.co/t/delete-old-logs/245268 "2020-08-17T15:39:26Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\
**Post date:** [August 17, 2020, 3:39pm UTC](https://discuss.elastic.co/t/delete-old-logs/245268/1 "2020-08-17T15:39:26Z")

</div>

Is it possible to have logs older than 3 month deleted? Or set a size limit? Is it possible to do so from Kibana?

Thanks ahead!

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 17, 2020, 9:59pm UTC](https://discuss.elastic.co/t/delete-old-logs/245268/2 "2020-08-17T21:59:17Z")

</div>

Which logs are you referring to here?

---

<div class="post-metadata">

**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\
**Post date:** [August 18, 2020, 7:04am UTC](https://discuss.elastic.co/t/delete-old-logs/245268/3 "2020-08-18T07:04:31Z")

</div>

Thanks for the response,

all the logs really. According to our compliance we need to keep 3 months of logs that the logging server recieves.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 18, 2020, 7:05am UTC](https://discuss.elastic.co/t/delete-old-logs/245268/4 "2020-08-18T07:05:39Z")

</div>

But do you mean the Elasticsearch logs? The source logs? The logs that are ingested into Elasticsearch?

---

<div class="post-metadata">

**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\
**Post date:** [August 18, 2020, 7:09am UTC](https://discuss.elastic.co/t/delete-old-logs/245268/5 "2020-08-18T07:09:04Z")

</div>

Thanks for the response,

I meant the logs from the remote servers. We plan on having about 50 remote server that would ship logs to elasticsearc (for example entries from /var/log/mysqld.log on each remote machine) Those need to be removed after 3 months on the logging server.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 18, 2020, 7:12am UTC](https://discuss.elastic.co/t/delete-old-logs/245268/6 "2020-08-18T07:12:00Z")

</div>

Ok, then that's outside the scope of the stack.

The linux syslog server has configurable retention. Otherwise a cronjob will work.

---

<div class="post-metadata">

**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\
**Post date:** [August 18, 2020, 7:26am UTC](https://discuss.elastic.co/t/delete-old-logs/245268/7 "2020-08-18T07:26:04Z")

</div>

But the logs are saved in Elasticstack, is there a command that would delete these logs that I can run in crontab once in three months? Or if I'm using the syslog log retention, what path do I need to give it to delete these entries?

Also, I asked this question in another forum and I was told that ILM can do that job. Is it made for something else?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 18, 2020, 7:28am UTC](https://discuss.elastic.co/t/delete-old-logs/245268/8 "2020-08-18T07:28:46Z")

</div>

It wasn't really clear from what you have answered sorry, which is why I asked the clarifying question 🙂

If you are talking about the logs _once they have been ingested into Elasticsearch_, then yes, ILM is the best thing for that.  
If you are talking about the logs _on the remote hosts_, you will need the other suggestion.

---

<div class="post-metadata">

**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\
**Post date:** [August 18, 2020, 10:36am UTC](https://discuss.elastic.co/t/delete-old-logs/245268/9 "2020-08-18T10:36:25Z")

</div>

Thanks for the response,

If I set Hot for 90 days, cold for 365 days and delete for 0. Does that mean that incoming logs would be on cold for a full year after and has been on hot for 3 month already? And after the year and 3 month get immediately deleted?

It says that it has 0 linked indiced. Can I link it to all indiced? I have an index named `cleandata` but I can't seem to find it there.

Also, are all logs in hot loaded into ram?

Sorry for all the questions, hope it's okay. These are the last one. Huge thanks ahead.

---

<div class="post-metadata">

**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\
**Post date:** [August 18, 2020, 11:00am UTC](https://discuss.elastic.co/t/delete-old-logs/245268/10 "2020-08-18T11:00:49Z")

</div>

To expand on me not seeing my index pattern (cleandata) in ILM,

Here are my index pattern in Kibana \> Discover : [https://i.imgur.com/Yygc0K5.png](https://i.imgur.com/Yygc0K5.png)  
This is what I see in the ILM settings: [https://i.imgur.com/wF68yZv.png](https://i.imgur.com/wF68yZv.png)

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 19, 2020, 5:04am UTC](https://discuss.elastic.co/t/delete-old-logs/245268/11 "2020-08-19T05:04:12Z")

</div>

> [@headtea](#):
>
> Also, are all logs in hot loaded into ram?

No, that is not how Elasticsearch works.

Do you have an index template for the `cleandata` index?

---

<div class="post-metadata">

**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\
**Post date:** [August 19, 2020, 9:00am UTC](https://discuss.elastic.co/t/delete-old-logs/245268/12 "2020-08-19T09:00:04Z")

</div>

Thanks for the response,

In Kibana \> Stack Management \> Index Management \> Index templates: I have created a new template named `cleandata` and gone with the defualt. I'm not sure it actually uses the `cleandata` indice. Is there a way of making the new index template use the `cleandata` indice?

This is a screenshot of my index management page: [https://i.imgur.com/9QcxbQM.png](https://i.imgur.com/9QcxbQM.png)

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 19, 2020, 9:27pm UTC](https://discuss.elastic.co/t/delete-old-logs/245268/13 "2020-08-19T21:27:06Z")

</div>

Ok that looks ok, do you have any `cleandata` indices? Were they created before or after the template was added?

---

<div class="post-metadata">

**Author:** ![headtea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/headtea/32/24271_2.png) [@headtea](https://discuss.elastic.co/u/headtea)\
**Post date:** [August 20, 2020, 7:30am UTC](https://discuss.elastic.co/t/delete-old-logs/245268/14 "2020-08-20T07:30:50Z")

</div>

Thanks for the response!

yes I have had the `cleandata` indice a lot before (in logstash config I have `index => "cleandata"`). I created the `cleandata` index yesterday and it doesn't seem to be connected to the `cleandata` indice.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 17, 2020, 7:30am UTC](https://discuss.elastic.co/t/delete-old-logs/245268/15 "2020-09-17T07:30:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
