# Logstash-plain.log is 17G in size

**URL:** <https://discuss.elastic.co/t/logstash-plain-log-is-17g-in-size/140643>\
**Category:** Logstash\
**Created:** [July 19, 2018, 1:02am UTC](https://discuss.elastic.co/t/logstash-plain-log-is-17g-in-size/140643 "2018-07-19T01:02:09Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![databeata](https://avatars.discourse-cdn.com/v4/letter/d/5e9695/32.png) [@databeata](https://discuss.elastic.co/u/databeata)\
**Post date:** [July 19, 2018, 1:02am UTC](https://discuss.elastic.co/t/logstash-plain-log-is-17g-in-size/140643/1 "2018-07-19T01:02:09Z")

</div>

Why is logstash-plain.log 17G in size?

ls -lh /path/logstash/path\_logs

-rw-r--r-- 1 root root 4.8M Jul 17 23:59 logstash-plain-2018-07-17.log  
-rw-r--r-- 1 root root 17G Jul 18 22:43 logstash-plain.log

cat /path/logstash/path\_logs/logstash-plain-2018-07-17.log

...  
[2018-07-17T23:59:49,973][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>"641401326", :\_index=\>"test", :\_type=\>"test", :\_routing=\>nil}, #LogStash::Event:0x7af1884b], :response=\>{"index"=\>{"\_index"=\>"test", "\_type"=\>"test", "\_id"=\>"641401326", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse [log\_date]", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Invalid format: "...""}}}}}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 19, 2018, 1:04am UTC](https://discuss.elastic.co/t/logstash-plain-log-is-17g-in-size/140643/2 "2018-07-19T01:04:38Z")

</div>

Probably because you have bad data going through your pipeline that isn't being accepted by Elasticsearch, so it's filling the log with the (relatively large) json response.

---

<div class="post-metadata">

**Author:** ![databeata](https://avatars.discourse-cdn.com/v4/letter/d/5e9695/32.png) [@databeata](https://discuss.elastic.co/u/databeata)\
**Post date:** [July 19, 2018, 1:07am UTC](https://discuss.elastic.co/t/logstash-plain-log-is-17g-in-size/140643/3 "2018-07-19T01:07:05Z")

</div>

Is it normal to have 17g file? At what size does it roll?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 19, 2018, 1:08am UTC](https://discuss.elastic.co/t/logstash-plain-log-is-17g-in-size/140643/4 "2018-07-19T01:08:04Z")

</div>

I am not sure how Logstash handles log rotation, sorry. Perhaps someone else can comment there.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 19, 2018, 1:22am UTC](https://discuss.elastic.co/t/logstash-plain-log-is-17g-in-size/140643/5 "2018-07-19T01:22:09Z")

</div>

Logstash uses log4j, so one would have to check /etc/logstash/log4j2.properties to see what the rolling policy on the appender is. In 6.3 the default is daily rotation plus size based rotation plus gzip of old logs. But that was just introduced. I would not be surprised if older versions used a plain DailyRollingFileAppender, so they just had one log per day.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 16, 2018, 1:22am UTC](https://discuss.elastic.co/t/logstash-plain-log-is-17g-in-size/140643/6 "2018-08-16T01:22:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.