Issues with kv filtering/ElasticSearch filtering

[jshepheard (Josh)]

Hello.
I am having a strange issue with ELK stack which I am struggling to get my head around. (ElasticSearch 6.5).

My logstash config contains:

kv {
 field_split => ", "
 value_split => "="
}

which is working fine, however, one field which has a space in it isn't quite working correctly.

an example message (unfiltered) contains device=First Second (data sanatized) - but in the filtered field of device, it shows as device=First and hasn't got the second part of the string.

The mapping type for this field in paticular is:

       "device" : {
           "type" : "keyword"
         },

So I believe that is correct. Can anybody help?
Thank you in advance.

[Badger]

@jshepheard

field_split => ", "

So fields are separated by either comma or space, which means Second is a field with no value, so it does not get stored.

[jshepheard (Josh)]

Thank you @Badger.

I did some more investigation and noted that the fields are splitting on both ", " or " " or ",".

Do you know of a way I can explicitly say, only split on ", "?

Thank you in advance.

[Jenni]

You need to change field_split to field_split_pattern.
https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html#plugins-filters-kv-field_split_pattern