# Syslog input filling disk

**URL:** <https://discuss.elastic.co/t/syslog-input-filling-disk/50286>\
**Category:** Logstash\
**Created:** [May 18, 2016, 3:05am UTC](https://discuss.elastic.co/t/syslog-input-filling-disk/50286 "2016-05-18T03:05:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ironbeast](https://avatars.discourse-cdn.com/v4/letter/i/b5a626/32.png) [@ironbeast](https://discuss.elastic.co/u/ironbeast)\
**Post date:** [May 18, 2016, 3:05am UTC](https://discuss.elastic.co/t/syslog-input-filling-disk/50286/1 "2016-05-18T03:05:52Z")

</div>

Hi All,

I have possible a dumb question... :-). We have 4 logstash servers running and are configured to receive syslog messages. But, the catch is that apparently logstash doesn't send the logs directly (parsed) to elasticsearch, but instead keeps them on the disk, causing off course the disk filling up.  
/srv/log/messages: 1.4G  
/srv/log/user.log: 1.4G

This: [https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/configuring-destinations-elasticsearch.html](https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/configuring-destinations-elasticsearch.html)... Is not a option, because of the different types (Solaris, Linux,...) Servers that are sending their logs to logstash.

So is there a way that the logs get to logstash, getting parsed and are send directly (like all the other logs) to elasticsearch?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 18, 2016, 3:37am UTC](https://discuss.elastic.co/t/syslog-input-filling-disk/50286/2 "2016-05-18T03:37:09Z")

</div>

Normally Logstash hardly uses any disk at all. Its own log should be mostly silent and only report problems. Who or what is producing /srv/log/{message,user}.log? Those are non-standard files so I can only assume that you yourself have configured Logstash to create them.

---

<div class="post-metadata">

**Author:** ![ironbeast](https://avatars.discourse-cdn.com/v4/letter/i/b5a626/32.png) [@ironbeast](https://discuss.elastic.co/u/ironbeast)\
**Post date:** [May 18, 2016, 3:45am UTC](https://discuss.elastic.co/t/syslog-input-filling-disk/50286/3 "2016-05-18T03:45:38Z")

</div>

Hi, thanks for your reply. This is my logstash config for syslog:  
input {  
tcp {  
port =\> 514  
type =\> syslog  
}  
udp {  
type =\> syslog  
port =\> 514  
}  
}  
filter {  
if [type] == "syslog" {  
grok {  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" }  
add\_field =\> ["received\_at", "%{@timestamp}"]  
add\_field =\> ["received\_from", "%{host}"]  
add\_tag =\> ["syslog"]  
}  
date {  
match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]  
}  
}  
}  
output {  
if "\_grokparsefailure" in [tags] {  
null {}  
}  
elasticsearch {  
hosts =\> ["elasticsearch1:9200"]  
index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}

All other log input goes directly to elasticsearch... Just the servers that are not using filebeat and are using syslog to forward their logs to logstash are stranded

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 18, 2016, 5:36am UTC](https://discuss.elastic.co/t/syslog-input-filling-disk/50286/4 "2016-05-18T05:36:08Z")

</div>

Again, who or what is creating the two files in /srv/log? If Logstash indeed is running with the configuration above it's not Logstash creating the files. What's in them?

If the point of your null output is top drop events with the `_grokparsefailure` tag it's not working. Either use a drop filter or something like this in the output section:

```auto
if "_grokparsefailure" not in [tags] {
  elasticsearch {
    ...
  }
}

```

---

<div class="post-metadata">

**Author:** ![ironbeast](https://avatars.discourse-cdn.com/v4/letter/i/b5a626/32.png) [@ironbeast](https://discuss.elastic.co/u/ironbeast)\
**Post date:** [May 18, 2016, 5:57am UTC](https://discuss.elastic.co/t/syslog-input-filling-disk/50286/5 "2016-05-18T05:57:58Z")

</div>

Ok, found it... It was a faulty configured rsyslog file... Thanks!  
And big thanks for the extra \_grokparsefailure!!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:57am UTC](https://discuss.elastic.co/t/syslog-input-filling-disk/50286/6 "2017-07-06T04:57:15Z")

</div>