# Parsing syslog from linux rsyslog **URL:** \ **Category:** Logstash\ **Created:** [September 28, 2015, 3:16pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267 "2015-09-28T15:16:46Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![cnozmn](https://avatars.discourse-cdn.com/v4/letter/c/9de053/32.png) [@cnozmn](https://discuss.elastic.co/u/cnozmn)\ **Post date:** [September 28, 2015, 3:16pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/1 "2015-09-28T15:16:46Z")
Hi, I setup ELK stack on my centos machine. In addition, I'm getting syslogs from rsyslog of another centos, So I can see it with "tcpdump" but I wanna see that on Kibana. I think my problem is "logstash.conf file". I couldn't configurate correctly. So how should I configurate my logstash.conf file? Are there any example ? I couldn't find it. Please some help. Thanks a lot for any interest. Best regards. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 28, 2015, 3:23pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/2 "2015-09-28T15:23:44Z")
It'd help if you could post what you have set in your config already. ---
**Author:** ![orgito](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/orgito/32/5051_2.png) [@orgito](https://discuss.elastic.co/u/orgito)\ **Post date:** [September 28, 2015, 3:43pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/3 "2015-09-28T15:43:28Z")
I simply configure rsyslog to forward data to logstash using RSYSLOG\_ForwardFormat template ``` action(type="omfwd" target="logstash-ip" port="51400" protocol="udp" template="RSYSLOG_ForwardFormat") ``` and my logstash.conf input is configured like that: ``` input { syslog { type => "syslog" port => 51400 } } ``` ---
**Author:** ![cnozmn](https://avatars.discourse-cdn.com/v4/letter/c/9de053/32.png) [@cnozmn](https://discuss.elastic.co/u/cnozmn)\ **Post date:** [September 28, 2015, 3:51pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/4 "2015-09-28T15:51:34Z")
Also I need to some filter conf? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 28, 2015, 5:16pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/5 "2015-09-28T17:16:26Z")
Here's an example from the Logstash documentation that should be very close to what you need: [https://www.elastic.co/guide/en/logstash/current/config-examples.html#\_processing\_syslog\_messages](https://www.elastic.co/guide/en/logstash/current/config-examples.html#_processing_syslog_messages) ---
**Author:** ![cnozmn](https://avatars.discourse-cdn.com/v4/letter/c/9de053/32.png) [@cnozmn](https://discuss.elastic.co/u/cnozmn)\ **Post date:** [September 28, 2015, 5:30pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/6 "2015-09-28T17:30:54Z")
I already tried that one. Unfortunately it doesn't work ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 28, 2015, 5:37pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/7 "2015-09-28T17:37:39Z")
If you can be a bit more specific than "it doesn't work" maybe someone will help you. - What, exactly, have you tried? - What result do you get? - What result did you expect? ---
**Author:** ![cnozmn](https://avatars.discourse-cdn.com/v4/letter/c/9de053/32.png) [@cnozmn](https://discuss.elastic.co/u/cnozmn)\ **Post date:** [September 28, 2015, 5:47pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/8 "2015-09-28T17:47:20Z")
Actually, I tried that conf; input { tcp { port =\> 514 type =\> syslog } udp { port =\> 514 type =\> syslog } } filter { if [type] == "syslog" { grok { match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" } add\_field =\> ["received\_at", "%{@timestamp}"] add\_field =\> ["received\_from", "%{host}"] } syslog\_pri { } date { match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } } output { elasticsearch { host =\> localhost } stdout { codec =\> rubydebug } } Whats wrong with me ? Also I looked with "tcpdump" and syslogs are coming. I can know that. But its not parsing.. I couldn't see on Kibana ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 28, 2015, 6:00pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/9 "2015-09-28T18:00:53Z")
But that's not quite the configuration from the example; you're using port 514 instead of port 5000. Unless you're running Logstash as root (or use a workaround) that won't work and Logstash should be complaining about this in the log. ---
**Author:** ![cnozmn](https://avatars.discourse-cdn.com/v4/letter/c/9de053/32.png) [@cnozmn](https://discuss.elastic.co/u/cnozmn)\ **Post date:** [September 28, 2015, 6:08pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/10 "2015-09-28T18:08:52Z")
I already changed LS\_USER from etc/sysconfig/logstash, I put LS\_USER=root , it was "LS\_USER=logstash". When I tried before this changing, service of logstash exited. But now service of logstash is runnning. You mean that its still a problem? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 28, 2015, 6:17pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/11 "2015-09-28T18:17:54Z")
Again, you need to read Logstash's logs. You may have to crank up the loglevel by adding `--verbose` or `--debug` to the Logstash command which also can be done via /etc/sysconfig/logstash. ---
**Author:** ![cnozmn](https://avatars.discourse-cdn.com/v4/letter/c/9de053/32.png) [@cnozmn](https://discuss.elastic.co/u/cnozmn)\ **Post date:** [September 28, 2015, 6:27pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/12 "2015-09-28T18:27:06Z")
Actually I didn't understand how to do that? Could you give me more details to what I should do exactly? I hope that I will solve my problem with your advice. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 28, 2015, 6:59pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/13 "2015-09-28T18:59:41Z")
Change the ``` #LS_OPTS="" ``` line to e.g. ``` LS_OPTS="--verbose" ``` ---
**Author:** ![cnozmn](https://avatars.discourse-cdn.com/v4/letter/c/9de053/32.png) [@cnozmn](https://discuss.elastic.co/u/cnozmn)\ **Post date:** [September 29, 2015, 12:09pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/14 "2015-09-29T12:09:03Z")
I tried that one too. But I still couldn't see on Kibana. Syslogs are not coming to Kibana ? Here I copied my file of "/etc/sysconfig/logstash" Please check for me, I don't know I need to change something more? , I'm confused about that. Thanks for your interest and helping me ``` ############################### # Default settings for logstash ############################### # Override Java location #JAVACMD=/usr/bin/java # Set a home directory #LS_HOME=/var/lib/logstash # Arguments to pass to logstash agent LS_OPTS="--verbose" # Arguments to pass to java #LS_HEAP_SIZE="500m" #LS_JAVA_OPTS="-Djava.io.tmpdir=$HOME" # pidfiles aren't used for upstart; this is for sysv users. #LS_PIDFILE=/var/run/logstash.pid # user id to be invoked as; for upstart: edit /etc/init/logstash.conf LS_USER=root # logstash logging #LS_LOG_FILE=/var/log/logstash/logstash.log #LS_USE_GC_LOGGING="true" # logstash configuration directory #LS_CONF_DIR=/etc/logstash/conf.d # Open file limit; cannot be overridden in upstart #LS_OPEN_FILES=16384 # Nice level #LS_NICE=19 # If this is set to 1, then when `stop` is called, if the process has # not exited within a reasonable time, SIGKILL will be sent next. # The default behavior is to simply log a message "program stop failed; still running" KILL_ON_STOP_TIMEOUT=0 ``` ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 29, 2015, 12:30pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/15 "2015-09-29T12:30:27Z")
And what do the Logstash logs contain after you've changed LS\_OPTS and restarted Logstash? You might also want to check whether Logstash is actually listing on port 514. Use e.g. `netstat` for that. ---
**Author:** ![cnozmn](https://avatars.discourse-cdn.com/v4/letter/c/9de053/32.png) [@cnozmn](https://discuss.elastic.co/u/cnozmn)\ **Post date:** [September 29, 2015, 1:01pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/16 "2015-09-29T13:01:54Z")
I checked "netstat" so port 514 is listening. Also when I looked "tcpdump", syslogs are coming. I can see that. My problem is parsing but I dont know why our conf doesn't work. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 29, 2015, 1:19pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/17 "2015-09-29T13:19:12Z")
Are the messages actually reaching Logstash? If you disable the elasticsearch output for now to simplify the system, are you getting output to stdout (probably connected to /var/log/logstash/logstash.stdout or similar)? What if you re-enable the elasticsearch output? ---
**Author:** ![cnozmn](https://avatars.discourse-cdn.com/v4/letter/c/9de053/32.png) [@cnozmn](https://discuss.elastic.co/u/cnozmn)\ **Post date:** [September 29, 2015, 7:40pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/18 "2015-09-29T19:40:45Z")
There is a lot of logs but why kibana doesn't show it? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 30, 2015, 6:19am UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/19 "2015-09-30T06:19:50Z")
That's what I'm trying to help you figure out, but if you don't answer my questions I can't help you. ---
**Author:** ![cnozmn](https://avatars.discourse-cdn.com/v4/letter/c/9de053/32.png) [@cnozmn](https://discuss.elastic.co/u/cnozmn)\ **Post date:** [September 30, 2015, 1:09pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267/20 "2015-09-30T13:09:16Z")
Yes, the messages are reaching Logstash. You said that "disable the elastic search" which means in the logstash.conf file, about output now ; ``` output{ elasticsearch { host => localhost } stdout { codec => rubydebug } } ``` You mean that changing like this? ``` output{ stdout { codec => rubydebug } } ``` after that trying again to kibana? or just disable elasticsearch.service like this ? `systemctl disable elasticsearch` [Next page](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267.md?page=2)