# \[SOLVED\] Logstash 5.4 Grok match partially working, can't figure why **URL:** \ **Category:** Logstash\ **Created:** [June 8, 2017, 7:11am UTC](https://discuss.elastic.co/t/solved-logstash-5-4-grok-match-partially-working-cant-figure-why/88667 "2017-06-08T07:11:50Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Sebastien\_Lelak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sebastien_lelak/32/75519_2.png) [@Sebastien\_Lelak](https://discuss.elastic.co/u/Sebastien_Lelak)\ **Post date:** [June 8, 2017, 7:11am UTC](https://discuss.elastic.co/t/solved-logstash-5-4-grok-match-partially-working-cant-figure-why/88667/1 "2017-06-08T07:11:50Z")
Greetings, For my first thread to you, I would like to submit an issue to your expertise. I have been using ELK stack (5.4) to gather data from our local radio station stream. I set up a Logstash file input which looks like this > ... > 12.255.255.187#Bose Monitoring Service#113369#5136220 > ... I have been using successfully the grok parser as follow: > ``` > filter > { > grok { > match => [ > "message", "%{IPORHOST:IP}#%{DATA:UserAgent}#%{NUMBER:TimeConnected}#%{NUMBER:ID}" > ] > } > } > > geoip { > source => "IP" > target => "geoip" > add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}" ] > add_field => ["[geoip][coordinates]", "%{[geoip][latitude]}" ] > } > > mutate { > convert => ["[geoip][coordinates]", "float"] > } > > useragent { > source => "UserAgent" > prefix=> "browser" > } > } > > ``` ... Everything was working fine, so I wanted more! I started appending our stream metadata to get what was listening to at a given time. It could be an ad, a radio show, a track from our playlist, a jingle, etc. The input file started to look like this: > ... > 138.255.255.224#WinampMPEG/5.50#72596#5164253#ADS > 144.255.255.70#VLC/2.1.4 LibVLC/2.1.4#14647#5215414#LINER > 78.255.255.203#fbxmms/1.0 FreeboxPlayer/6.0 (like AppleWebKit)#5591#5218475#BOBBY'S-AWESOME\_RADIO SHOW > 78.196.255.255#Lavf53.32.100#2857#5219883#HOURLY JINGLE > 92.255.255.93#VLC/2.2.6 LibVLC/2.2.6#2653#5220001#ARTIST WITH SPACES#SONGTITLE WITH SPACES > ... So, I set up my grok filter accordingly, or at least I tried to: > filter > { > grok { > match =\> { "message" =\> [ > "%{IPORHOST:IP}#%{DATA:UserAgent}#%{NUMBER:TimeConnected}#%{NUMBER:ID}#%{DATA:Artist}#%{DATA:SongTitle}", > "%{IPORHOST:IP}#%{DATA:UserAgent}#%{NUMBER:TimeConnected}#%{NUMBER:ID}#%{DATA:BAJingleAdOrShow}", > "%{IPORHOST:IP}#%{DATA:UserAgent}#%{NUMBER:TimeConnected}#%{NUMBER:ID}" > ] > } > } > ... > } Here comes my issue: from that filter configuration, I see in Kibana that I got a new indexed field: Artist. But so far, it fails to create/add the SongTitle one. I tried switching DATA to SPACE, but it didn't do anything. Our radio schedule won't air any radio show until this evening, so I figured that's why the BAJingleAdOrShow doesn't show up yet in Kabana field's list, so I do not worry with that yet. Why would the file parsing filter partially work? How could it create/update (in elasticsearch) the Artist field without creating/updating the SongTitle field at the same time? Best Regards from France, ---
**Author:** ![Sebastien\_Lelak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sebastien_lelak/32/75519_2.png) [@Sebastien\_Lelak](https://discuss.elastic.co/u/Sebastien_Lelak)\ **Post date:** [June 8, 2017, 6:20pm UTC](https://discuss.elastic.co/t/solved-logstash-5-4-grok-match-partially-working-cant-figure-why/88667/2 "2017-06-08T18:20:41Z")
Good evening, As it turned out, the second possibility "%{IPORHOST:IP}#%{DATA:UserAgent}#%{NUMBER:TimeConnected}#%{NUMBER:ID}#%{DATA:BAJingleAdOrShow} isn't parsed, and it falls back to "%{IPORHOST:IP}#%{DATA:UserAgent}#%{NUMBER:TimeConnected}#%{NUMBER:ID}" ; I still doesn't retrieve the SongTitle as a indexed field. So to sum it up : > 92.255.255.93#VLC/2.2.6 LibVLC/2.2.6#2653#5220001#ARTIST WITH SPACES#SONGTITLE WITH SPACES this is parsed with "%{IPORHOST:IP}#%{DATA:UserAgent}#%{NUMBER:TimeConnected}#%{NUMBER:ID}#%{DATA:Artist}#%{DATA:SongTitle}" but the SongTitle isn't shown within Kibana (but Artist is) > 138.255.255.224#WinampMPEG/5.50#72596#5164253#ADS > 144.255.255.70#VLC/2.1.4 LibVLC/2.1.4#14647#5215414#LINER > 78.255.255.203#fbxmms/1.0 FreeboxPlayer/6.0 (like AppleWebKit)#5591#5218475#BOBBY'S-AWESOME\_RADIO SHOW > 78.196.255.255#Lavf53.32.100#2857#5219883#HOURLY JINGLE those lines aren't parsed with "%{IPORHOST:IP}#%{DATA:UserAgent}#%{NUMBER:TimeConnected}#%{NUMBER:ID}#%{DATA:BAJingleAdOrShow} since the field BAJingleAdOrShow isn't shown within Kibana (even after refreshing the index in Management) but the data are still output within elasticsearch with the "%{IPORHOST:IP}#%{DATA:UserAgent}#%{NUMBER:TimeConnected}#%{NUMBER:ID}" pattern. I can't seem to figure what is going wrong here ; do you have any clue? Best Regards ---
**Author:** ![Sebastien\_Lelak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sebastien_lelak/32/75519_2.png) [@Sebastien\_Lelak](https://discuss.elastic.co/u/Sebastien_Lelak)\ **Post date:** [June 14, 2017, 2:26pm UTC](https://discuss.elastic.co/t/solved-logstash-5-4-grok-match-partially-working-cant-figure-why/88667/3 "2017-06-14T14:26:33Z")
Hello there, So I managed to get a config that works, so I am posting it right there before updating this thread as saolved 🙂 Thank you for your concern! > filter > { > grok { > match =\> [ > "message", "%{IPORHOST:IP}#%{DATA:UserAgent}#%{NUMBER:TimeConnected}#%{NUMBER:ID}#%{DATA:Artist}#%{USER:SongTitle}", > "message", "%{IPORHOST:IP}#%{DATA:UserAgent}#%{NUMBER:TimeConnected}#%{NUMBER:ID}#%{USER:BAJingleAdOrShow}", > "message", "%{IPORHOST:IP}#%{DATA:UserAgent}#%{NUMBER:TimeConnected}#%{NUMBER:ID}" > ] > } > } ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 12, 2017, 2:32pm UTC](https://discuss.elastic.co/t/solved-logstash-5-4-grok-match-partially-working-cant-figure-why/88667/4 "2017-07-12T14:32:57Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.