# Logstash grok filter with syslog **URL:** \ **Category:** Logstash\ **Created:** [October 30, 2016, 8:09pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-with-syslog/64386 "2016-10-30T20:09:09Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![tolasto](https://avatars.discourse-cdn.com/v4/letter/t/da6949/32.png) [@tolasto](https://discuss.elastic.co/u/tolasto)\ **Post date:** [October 30, 2016, 8:09pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-with-syslog/64386/1 "2016-10-30T20:09:09Z")
Hi, I'm tryin to get a filter for this logfile with logstash: `2016-10-30T13:23:47+01:00 router.lan pppd[12566]: local IP address 1.2.3.4` The Grok debugger can resolve the fields nicely with this expression: `%{DATE:datum}T%{TIME:time}%{ISO8601_TIMEZONE:timezone} %{HOSTNAME:hostname} %{WORD:service}%{GREEDYDATA:id fields} %{IP:wanip}` What I would like to get working with your help(after trying unsuccessfully for a day) is transfering the fields recognized by grok into elasticsearch via a logstash config for being able to filter in kibana for e.g. wanip Hope you can help 🙂 The logstash looks like this at the moment: ``` input { file { path => "/var/log/rsyslog/router1.log" start_position => "beginning" type => "routerlog" } } filter { if [type] == "routerlog" { grok { match => { "message" => "%{DATE:datum} <<<>>>>>} } } } output { elasticsearch { hosts => ["localhost:9200"] } } ``` ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 31, 2016, 6:51am UTC](https://discuss.elastic.co/t/logstash-grok-filter-with-syslog/64386/2 "2016-10-31T06:51:06Z")
> [@tolasto](#): > > output { > elasticsearch { > hosts =\> ["localhost:9200"] > } > } Does this not work? What happens? What version are you on? ---
**Author:** ![tolasto](https://avatars.discourse-cdn.com/v4/letter/t/da6949/32.png) [@tolasto](https://discuss.elastic.co/u/tolasto)\ **Post date:** [October 31, 2016, 7:08am UTC](https://discuss.elastic.co/t/logstash-grok-filter-with-syslog/64386/3 "2016-10-31T07:08:34Z")
It's working if I remove the filter section or just have one match in the filter. But then it's just recognizing the timestamp and some fields I don't need. I would like to have it recognizing the IP and fqdn. How do I send the file nicely parsed (like grok does it easily 🙂 ) to elasticsearch? I'm on Version 5.0.0 with all components. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 31, 2016, 8:30pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-with-syslog/64386/4 "2016-10-31T20:30:11Z")
Instead of `%{DATE:datum}T%{TIME:time}%{ISO8601_TIMEZONE:timezone}` use `%{TIMESTAMP_ISO8601:timestamp}`. For the sake of the date filter you'll want to have the full timestamp in a single field anyway. The `DATE` pattern doesn't match yyyy-mm-dd dates. ---
**Author:** ![tolasto](https://avatars.discourse-cdn.com/v4/letter/t/da6949/32.png) [@tolasto](https://discuss.elastic.co/u/tolasto)\ **Post date:** [October 31, 2016, 9:46pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-with-syslog/64386/5 "2016-10-31T21:46:50Z")
Thanks for the tip 🙂 Yes I probably just need one field for date. My main problem still is: How do I get everything nicely filtered (with the grok expressions I have) into elasticsearch and Kibana? I just want to get nicely searchable data in kibana - or do I apply the grok filter somewhere in kibana? Sorry, noob questions 🙂 I hope I can start digging through bigger logs soon... ---
**Author:** ![MGG](https://avatars.discourse-cdn.com/v4/letter/m/c68b51/32.png) [@MGG](https://discuss.elastic.co/u/MGG)\ **Post date:** [November 1, 2016, 2:41am UTC](https://discuss.elastic.co/t/logstash-grok-filter-with-syslog/64386/6 "2016-11-01T02:41:56Z")
Start with this -- %{TIMESTAMP\_ISO8601:timestamp} %{DATA:router} %{DATA:proc}: %{GREEDYDATA:msg} If you wish to play with grok a bit more and refine your filter, you can use this -- [http://grokconstructor.appspot.com/do/match#result](http://grokconstructor.appspot.com/do/match#result) ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 1, 2016, 6:31am UTC](https://discuss.elastic.co/t/logstash-grok-filter-with-syslog/64386/7 "2016-11-01T06:31:15Z")
> My main problem still is: How do I get everything nicely filtered (with the grok expressions I have) into elasticsearch and Kibana? If you extract the fields with the grok filter like you're doing in your first example (where you have `%{HOSTNAME:hostname}` etc) they will end up in Elasticsearch and will therefore be available in Kibana. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 1, 2016, 6:34am UTC](https://discuss.elastic.co/t/logstash-grok-filter-with-syslog/64386/8 "2016-11-01T06:34:44Z")
> ``` > %{TIMESTAMP_ISO8601:timestamp} %{DATA:router} %{DATA:proc}\: %{GREEDYDATA:msg} > > ``` Using more than one DATA or GREEDYDATA in the same expression easily leads to weird matches. I instead suggest use of standard patterns for syslog messages: ``` %{TIMESTAMP_ISO8601:timestamp} %{SYSLOGHOST:host} %{SYSLOGPROG}: %{GREEDYDATA:msg} ``` > ---
**Author:** ![tolasto](https://avatars.discourse-cdn.com/v4/letter/t/da6949/32.png) [@tolasto](https://discuss.elastic.co/u/tolasto)\ **Post date:** [November 2, 2016, 8:02pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-with-syslog/64386/9 "2016-11-02T20:02:37Z")
Thanks all! I went with the last one from magnus. It has no Field for IP but I'll get there 🙂 It's a nice tool with a lot of potential. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:31am UTC](https://discuss.elastic.co/t/logstash-grok-filter-with-syslog/64386/10 "2017-07-06T04:31:27Z")