# Still problems with grok parsing **URL:** \ **Category:** Logstash\ **Created:** [November 14, 2018, 3:55pm UTC](https://discuss.elastic.co/t/still-problems-with-grok-parsing/156703 "2018-11-14T15:55:55Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![heskez](https://avatars.discourse-cdn.com/v4/letter/h/ecae2f/32.png) [@heskez](https://discuss.elastic.co/u/heskez)\ **Post date:** [November 14, 2018, 3:55pm UTC](https://discuss.elastic.co/t/still-problems-with-grok-parsing/156703/1 "2018-11-14T15:55:55Z")
Hi there, I've still problems parsing with grok. This is the raw data coming in: ``` 1: date=2018-10-17 time=03:19:48 logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.10.1.150 srcport=52120 srcintf="internal3" dstip=8.8.8.8 dstport=53 dstintf="wan1" poluuid=45245042-ccf9-51e6-28f5-f8fa6131fcdf sessionid=22161176 proto=17 action=dns policyid=1 appcat="unscanned" crscore=5 craction=262144 crlevel=low ``` 2: date=2018-10-17 time=03:14:48 logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.10.1.150 srcport=49390 srcintf="internal3" dstip=8.8.8.8 dstport=53 dstintf="wan1" poluuid=45245042-ccf9-51e6-28f5-f8fa6131fcdf sessionid=22160929 proto=17 action=dns policyid=1 appcat="unscanned" crscore=5 craction=262144 crlevel=low 3: date=2018-10-17 time=03:13:28 logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.10.1.160 srcport=35729 srcintf="internal3" dstip=8.8.8.8 dstport=53 dstintf="wan1" poluuid=45245042-ccf9-51e6-28f5-f8fa6131fcdf sessionid=22160852 proto=17 action=dns policyid=1 appcat="unscanned" crscore=5 craction=262144 crlevel=low 4: date=2018-10-17 time=03:09:47 logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.10.1.150 srcport=33107 srcintf="internal3" dstip=8.8.8.8 dstport=53 dstintf="wan1" poluuid=45245042-ccf9-51e6-28f5-f8fa6131fcdf sessionid=22160715 proto=17 action=dns policyid=1 appcat="unscanned" crscore=5 craction=262144 crlevel=low 5: date=2018-10-17 time=03:04:47 logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.10.1.150 srcport=47123 srcintf="internal3" dstip=8.8.8.8 dstport=53 dstintf="wan1" poluuid=45245042-ccf9-51e6-28f5-f8fa6131fcdf sessionid=22160462 proto=17 action=dns policyid=1 appcat="unscanned" crscore=5 craction=262144 crlevel=low 6: date=2018-10-17 time=02:59:48 logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.10.1.150 srcport=40293 srcintf="internal3" dstip=8.8.8.8 dstport=53 dstintf="wan1" poluuid=45245042-ccf9-51e6-28f5-f8fa6131fcdf sessionid=22160240 proto=17 action=dns policyid=1 appcat="unscanned" crscore=5 craction=262144 crlevel=low 7: date=2018-10-17 time=02:58:00 logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.10.1.10 srcport=47648 srcintf="internal3" dstip=8.8.8.8 dstport=53 dstintf="wan1" poluuid=45245042-ccf9-51e6-28f5-f8fa6131fcdf sessionid=22160098 proto=17 action=dns policyid=1 appcat="unscanned" crscore=5 craction=262144 crlevel=low 8: date=2018-10-17 time=02:54:48 logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.10.1.150 srcport=51858 srcintf="internal3" dstip=8.8.8.8 dstport=53 dstintf="wan1" poluuid=45245042-ccf9-51e6-28f5-f8fa6131fcdf sessionid=22159984 proto=17 action=dns policyid=1 appcat="unscanned" crscore=5 craction=262144 crlevel=low 9: date=2018-10-17 time=02:49:47 logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.10.1.150 srcport=49577 srcintf="internal3" dstip=8.8.8.8 dstport=53 dstintf="wan1" poluuid=45245042-ccf9-51e6-28f5-f8fa6131fcdf sessionid=22159743 proto=17 action=dns policyid=1 appcat="unscanned" crscore=5 craction=262144 crlevel=low 10: date=2018-10-17 time=02:45:58 logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.10.1.160 srcport=60164 srcintf="internal3" dstip=8.8.8.8 dstport=53 dstintf="wan1" poluuid=45245042-ccf9-51e6-28f5-f8fa6131fcdf sessionid=22159509 proto=17 action=dns policyid=1 appcat="unscanned" crscore=5 craction=262144 crlevel=low These are the error messages: failure\_grok\_fortigate, \_dateparsefailure host: ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 14, 2018, 4:04pm UTC](https://discuss.elastic.co/t/still-problems-with-grok-parsing/156703/2 "2018-11-14T16:04:38Z")
I would recommend using a kv filter instead of grok if all your logs follow the same format. ---
**Author:** ![heskez](https://avatars.discourse-cdn.com/v4/letter/h/ecae2f/32.png) [@heskez](https://discuss.elastic.co/u/heskez)\ **Post date:** [November 19, 2018, 10:54am UTC](https://discuss.elastic.co/t/still-problems-with-grok-parsing/156703/3 "2018-11-19T10:54:11Z")
Thanks for your answer! What would be the kv filter looking at this raw data? I now have this configured: ``` input { udp { port => 5514 type => syslog } tcp { port => 5514 type => syslog } } filter { grok { patterns_dir => ["/etc/logstash/patterns"] match => ["message", "%{SYSLOG5424LINE}"] overwrite => ["message"] tag_on_failure => ["failure_grok_fortigate"] } kv { } if [msg] { mutate { replace => ["message", "%{msg}"] } } mutate { add_field => ["logTimestamp", "%{date} %{time}"] add_field => ["loglevel", "%{level}"] replace => ["fortigate_type", "%{type}"] replace => ["fortigate_subtype", "%{subtype}"] remove_field => ["msg", "type", "level", "date", "time"] } date { locale => "en" match => ["logTimestamp", "YYYY-MM-dd HH:mm:ss"] remove_field => ["logTimestamp", "year", "month", "day", "time", "date"] add_field => ["type", "syslog"] } } output { elasticsearch { hosts => ["localhost:9200"] index => "logstash-syslog" } stdout { codec => rubydebug } } ``` ---
**Author:** ![heskez](https://avatars.discourse-cdn.com/v4/letter/h/ecae2f/32.png) [@heskez](https://discuss.elastic.co/u/heskez)\ **Post date:** [November 20, 2018, 6:36pm UTC](https://discuss.elastic.co/t/still-problems-with-grok-parsing/156703/4 "2018-11-20T18:36:30Z")
Eh anyone? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 18, 2018, 6:36pm UTC](https://discuss.elastic.co/t/still-problems-with-grok-parsing/156703/5 "2018-12-18T18:36:32Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.