# How to clean logs generated by Logstash?

**URL:** <https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548>\
**Category:** Logstash\
**Created:** [April 27, 2016, 12:27pm UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548 "2016-04-27T12:27:16Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![djvidov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djvidov/32/6135_2.png) [@djvidov](https://discuss.elastic.co/u/djvidov)\
**Post date:** [April 27, 2016, 12:27pm UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/1 "2016-04-27T12:27:16Z")

</div>

Today my logstash stops to work because it's log folder was full with old logs since 2015.

There is a setting in logstash to delete logs older that x days?

Thanks,  
Ovidiu

---

<div class="post-metadata">

**Author:** ![Gaurav\_Harsola](https://avatars.discourse-cdn.com/v4/letter/g/ecc23a/32.png) [@Gaurav\_Harsola](https://discuss.elastic.co/u/Gaurav_Harsola)\
**Post date:** [April 27, 2016, 12:57pm UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/2 "2016-04-27T12:57:38Z")

</div>

@djvidov  
As per understanding you have to remove .sincedb file .

Thanks  
Gaurav

---

<div class="post-metadata">

**Author:** ![djvidov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djvidov/32/6135_2.png) [@djvidov](https://discuss.elastic.co/u/djvidov)\
**Post date:** [April 27, 2016, 1:44pm UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/3 "2016-04-27T13:44:25Z")

</div>

hi Gaurav,

This is fist time when I hear about this file and I didn't find it into logstash. Where I should find it? I have installed logstash 2.1.

I have many files like: logstash.log\_YYYYMMDDhhmmss.gz starting from logstash.log\_20151115123224.gz ... until ... logstash.log\_20160427153221.gz and i don't want to keep all these files, for me it's enough to have only last 15 days.

Thanks,  
Ovidiu

---

<div class="post-metadata">

**Author:** ![Gaurav\_Harsola](https://avatars.discourse-cdn.com/v4/letter/g/ecc23a/32.png) [@Gaurav\_Harsola](https://discuss.elastic.co/u/Gaurav_Harsola)\
**Post date:** [April 27, 2016, 1:57pm UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/4 "2016-04-27T13:57:26Z")

</div>

By default it is under root direcotry or your home directory .Better u can  
search using command find / -name '.sincedb\*'  
Thanks  
Gaurav

---

<div class="post-metadata">

**Author:** ![djvidov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djvidov/32/6135_2.png) [@djvidov](https://discuss.elastic.co/u/djvidov)\
**Post date:** [April 27, 2016, 2:09pm UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/5 "2016-04-27T14:09:57Z")

</div>

I have used this command but I still find nothing.

Thanks,  
Ovidiu

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 28, 2016, 5:33am UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/6 "2016-04-28T05:33:02Z")

</div>

> As per understanding you have to remove .sincedb file .

No, the OP's problem has nothing to do with sincedb.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [April 28, 2016, 5:35am UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/7 "2016-04-28T05:35:03Z")

</div>

> I have many files like: logstash.log\_YYYYMMDDhhmmss.gz starting from logstash.log\_20151115123224.gz ... until ... logstash.log\_20160427153221.gz and i don't want to keep all these files, for me it's enough to have only last 15 days.

How are these files created? Logstash itself doesn't name files this way and none of the log rotation tools I know about name files in this manner either.

Anyway, Logstash doesn't have any log rotation functionality at all so any purging of old rotated files needs to be managed outside of Logstash.

---

<div class="post-metadata">

**Author:** ![djvidov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djvidov/32/6135_2.png) [@djvidov](https://discuss.elastic.co/u/djvidov)\
**Post date:** [April 28, 2016, 7:58am UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/8 "2016-04-28T07:58:39Z")

</div>

Hello Magnus,

I have no idea how these files are created. These files are into logstash logs folder. I was looking to .config file that start logstash and I don't see any dump to file but probably "stdout { codec =\> rubydebug }" from output generate this file. I will try to remove it to see if something will change. 🙂

Thanks,  
Ovidiu

---

<div class="post-metadata">

**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\
**Post date:** [April 28, 2016, 11:37am UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/10 "2016-04-28T11:37:23Z")

</div>

If on a linux box, check to see if you've got eg. a /etc/logrotate.d/logstash file, as it seems your logstash.log is logrotated as magnus say this is handled outside logstash.

I've got this:

> # cat /etc/logrotate.d/logstash
> 
> /var/log/logstash/_.log /var/log/logstash/_.err /var/log/logstash/\*.stdout {  
> daily  
> rotate 7  
> copytruncate  
> compress  
> delaycompress  
> missingok  
> notifempty  
> }

PS! .sincedb is state DB for filebeat and got nothing to do with logstash as such expect filebeat might send data to a logstash instance(s)

---

<div class="post-metadata">

**Author:** ![djvidov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djvidov/32/6135_2.png) [@djvidov](https://discuss.elastic.co/u/djvidov)\
**Post date:** [April 28, 2016, 12:19pm UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/11 "2016-04-28T12:19:18Z")

</div>

Hi Steffen,  
I've check on my elk server and I have similar settings:

> ```
> [root@elkst logrotate.d]# cat logstash
> /var/log/logstash/*.log /var/log/logstash/*.err /var/log/logstash/*.stdout {
> daily
> rotate 7
> copytruncate
> compress
> delaycompress
> missingok
> notifempty
> }
> 
> ```

I'll try to add my files: logstash.log\_20151115123224.gz into logrotate system.

Thank you for hint,  
Ovidiu

---

<div class="post-metadata">

**Author:** ![stefws](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefws/32/6442_2.png) [@stefws](https://discuss.elastic.co/u/stefws)\
**Post date:** [April 28, 2016, 12:46pm UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/12 "2016-04-28T12:46:14Z")

</div>

You should not add logstash.log\_\*gz files to logrotate as they are created by logrotate originally.

Better check if your logrotate is scheduled properly by cron, check 'man logrotate' and 'man cron' if not familiar with these and/or Google linux logrotate.

---

<div class="post-metadata">

**Author:** ![djvidov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djvidov/32/6135_2.png) [@djvidov](https://discuss.elastic.co/u/djvidov)\
**Post date:** [May 4, 2016, 8:35am UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/13 "2016-05-04T08:35:13Z")

</div>

Ok, i'm not familiar with linux but I'll search.

thanks. 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:59am UTC](https://discuss.elastic.co/t/how-to-clean-logs-generated-by-logstash/48548/14 "2017-07-06T04:59:27Z")

</div>


