# Parsing KV filter is not working **URL:** \ **Category:** Logstash\ **Created:** [April 3, 2019, 12:30pm UTC](https://discuss.elastic.co/t/parsing-kv-filter-is-not-working/175192 "2019-04-03T12:30:02Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Loup](https://avatars.discourse-cdn.com/v4/letter/l/51bf81/32.png) [@Loup](https://discuss.elastic.co/u/Loup)\ **Post date:** [April 3, 2019, 12:30pm UTC](https://discuss.elastic.co/t/parsing-kv-filter-is-not-working/175192/1 "2019-04-03T12:30:02Z")
Hi, I need to **parse each field** from a McAfee WebGateway like having a field **url, ip source, destination**... Can you **please** help me to **understand my mistakes** with **kv** and **grok**? I do not success to make the **kv** works. Here is what I did: - **Log exemple** : "\<30\>Apr 3 14:15:55 MWG05 mwg: [03/Apr/2019:14:15:55 +0200]#jean-bombeur#192.168.1.1#13.117.5.81#90#200#TCP\_MISS\_RELOAD#GET [http://api.bing.com/qsml.aspx?query=troll&maxwidth=32765&rowheight=20§ionHeight=160&FORM=IESS02&market=fr-FR](http://api.bing.com/qsml.aspx?query=troll&maxwidth=32765&rowheight=20§ionHeight=160&FORM=IESS02&market=fr-FR) HTTP/1.1# #Search Engines, Internet Services# #Minimal Risk# #text/xml#1036#1315#Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko# #-# #-# #0# #0# #-# #-# #-# #GRP-INTERN#" - **part** of the **Logstash.conf** ```auto else if [host] in ["192.168.10.1", "192.168.10.2"] { # mcafee grok { match => ["message", "%{SYSLOG5424PRI}%{GREEDYDATA:data}"] add_tag => ["REMARQUABLE"] } kv { source => "data" field_split => "#" } } ``` - Here is the **actual result** ```auto { "host" => "192.168.10.1", "data" => "Apr 3 14:15:55 MWG05 mwg: [03/Apr/2019:14:15:55 +0200]#jean-bombeur#192.168.1.1#13.117.5.81#90#200#TCP_MISS_RELOAD#GET http://api.bing.com/qsml.aspx?query=troll&maxwidth=32765&rowheight=20§ionHeight=160&FORM=IESS02&market=fr-FR HTTP/1.1# #Search Engines, Internet Services# #Minimal Risk# #text/xml#1036#1315#Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko# #-# #-# #0# #0# #-# #-# #-# #GRP-INTERN#", "GET http://api.bing.com/qsml.aspx?query" => "troll&maxwidth=32765&rowheight=20§ionHeight=160&FORM=IESS02&market=fr-FR HTTP/1.1", "message" => "<30>Apr 3 14:15:55 MWG05 mwg: [03/Apr/2019:14:15:55 +0200]#jean-bombeur#192.168.1.1#13.17.5.81#90#200#TCP_MISS_RELOAD#GET http://api.bing.com/qsml.aspx?query=troll&maxwidth=32765&rowheight=20§ionHeight=160&FORM=IESS02&market=fr-FR HTTP/1.1# #Search Engines, Internet Services# #Minimal Risk# #text/xml#1036#1315#Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko# #-# #-# #0# #0# #-# #-# #-# #GRP-INTERN#", "type" => "mcafee", "@version" => "1", "@timestamp" => 2019-04-03T12:01:19.026Z, "tags" => [ [0] "REMARQUABLE" ], "syslog5424_pri" => "30" } ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 3, 2019, 2:50pm UTC](https://discuss.elastic.co/t/parsing-kv-filter-is-not-working/175192/2 "2019-04-03T14:50:32Z")
data does not appear to be key/value data. What output are you expecting? ---
**Author:** ![Loup](https://avatars.discourse-cdn.com/v4/letter/l/51bf81/32.png) [@Loup](https://discuss.elastic.co/u/Loup)\ **Post date:** [April 4, 2019, 3:23pm UTC](https://discuss.elastic.co/t/parsing-kv-filter-is-not-working/175192/3 "2019-04-04T15:23:47Z")
> [@Badger](#): > > key/value data Of course, I should have seen it. Sorry for this mistake and **thank you** for your time. Since I can modify the log format from McAfee, I did this (it looks fine to me but if you have any idea to improve, feel free to share it): **Source log** \<30\>Apr 4 16:59:41 MGW02 mwg: #date|[04/Apr/2019:16:59:41 +0200]#account|jean-bombeur#srcip|192.168.1.1#dstip|172.64.238.38#timing\_trans|44#statuscode|301#cache|TCP\_MISS\_RELOAD#request|HEAD [Greenshot](http://getgreenshot.org/project-feed/) HTTP/1.1# #Shareware/Freeware#reputation|Minimal Risk# #-#331#559#-# #-#virus\_name|-# #0#block\_id|0# #-# #-# #-#rule\_set|GRP-INTERN# **Part** of the **logstash.conf** ```auto else if [host] in ["192.168.10.1", "192.168.10.2","192.168.20.1", "192.168.20.2",] { # mcafee grok { match => ["message", "%{SYSLOG5424PRI}%{GREEDYDATA:data}"] } kv { source => "data" field_split => "#" value_split => "|" } mutate { remove_field => ["data"] } if [host] in ["192.168.10.1", "192.168.10.2"] { mutate { add_field => { "country" => "Germany" } } } if [host] in ["192.168.20.1", "192.168.20.2"] { mutate { add_field => { "country" => "USA" } } } } ``` **Actual result** ```auto { "srcip" => "192.168.1.1", "virus_name" => "-", "block_id" => "0", "message" => "<30>Apr 4 16:59:41 MGW02 mwg: #date|[04/Apr/2019:16:59:41 +0200]#account|jean-bombeur#srcip|192.168.1.1#dstip|172.64.238.38#timing_trans|44#statuscode|301#cache|TCP_MISS_RELOAD#request|HEAD http://getgreenshot.org/project-feed/ HTTP/1.1# #Shareware/Freeware#reputation|Minimal Risk# #-#331#559#-# #-#virus_name|-# #0#block_id|0# #-# #-# #-#rule_set|GRP-INTERN#", "date" => "04/Apr/2019:16:59:41 +0200", "cache" => "TCP_MISS_RELOAD", "host" => "192.168.10.1", "type" => "mcafee", "statuscode" => "301", "syslog5424_pri" => "30", "dstip" => "172.64.238.38", "request" => "HEAD http://getgreenshot.org/project-feed/ HTTP/1.1", "reputation" => "Minimal Risk", "timing_trans" => "44", "account" => "jean-bombeur", "rule_set" => "GRP-INTERN", "country" => "Germany", "@timestamp" => 2019-04-04T14:45:02.605Z, "@version" => "1" } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 2, 2019, 3:23pm UTC](https://discuss.elastic.co/t/parsing-kv-filter-is-not-working/175192/4 "2019-05-02T15:23:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.