# \[ES7.0.1\] Field 'cn'(keyword&text type) is not extracted as expected from message field **URL:** \ **Category:** Logstash\ **Created:** [September 16, 2019, 2:17am UTC](https://discuss.elastic.co/t/es7-0-1-field-cn-keyword-text-type-is-not-extracted-as-expected-from-message-field/199560 "2019-09-16T02:17:58Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![cheriemilk](https://avatars.discourse-cdn.com/v4/letter/c/c37758/32.png) [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Post date:** [September 16, 2019, 2:17am UTC](https://discuss.elastic.co/t/es7-0-1-field-cn-keyword-text-type-is-not-extracted-as-expected-from-message-field/199560/1 "2019-09-16T02:17:58Z")
Hi Team, in prod elk, there's below message with containing the value of cn field is GRIFOLS, S.A. > 2019-09-11 17:29:30,930 module=SCM fa=TS at=SCM.TS.LIST\_SAVED\_SEARCH si=566F57EC24877AC67B8D36655D69D490.vsa3029283 ci=Grifols **cn=GRIFOLS, S.A.** cs=dc5prd\_STOCKPM8601. pi=dbPool2 ui=bcwld locale=en\_US ktf1=E But it was parsed to below, and string 'S.A.' is lost. ` > cn=GRIFOLS, ` How to correct this? Below is the logstash configuration for filter part > ``` > filter { > grok { > match=> { > "message" =>"%{TIMESTAMP_ISO8601:logdate} " > } > } > > kv { > field_split => " " > value_split => "=" > trim_key => " " > trim_value => " " > include_brackets => false > > } > > > ruby { > code => " > hash = event.to_hash > hash.each do |k, v| > if(v != nil && v.kind_of?(String) && v.length > 2 && v[0,1] == '[' && v[v.length-1,1] == ']') > event.set(k, v[1, v.length-2].split(',')) > end > end > " > } > > date { > match => ["logdate", "yyyy-MM-dd HH:mm:ss,SSS"] > target => "@timestamp" > } > } > > ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 16, 2019, 3:39pm UTC](https://discuss.elastic.co/t/es7-0-1-field-cn-keyword-text-type-is-not-extracted-as-expected-from-message-field/199560/2 "2019-09-16T15:39:28Z")
In that specific case you can fix it by replacing field\_split with ``` field_split_pattern => "[^,] " ``` ---
**Author:** ![cheriemilk](https://avatars.discourse-cdn.com/v4/letter/c/c37758/32.png) [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Post date:** [September 18, 2019, 7:06am UTC](https://discuss.elastic.co/t/es7-0-1-field-cn-keyword-text-type-is-not-extracted-as-expected-from-message-field/199560/3 "2019-09-18T07:06:09Z")
Hi Badger, That actually doesn't work for all scenarios Here is the source message of kv pair. > ``` > module=SCM fa=PP at=SCM.PP.GENERATE si=E02F07592081B6A58F7FFC3DE035D62B ci=JShenSCM **cn=GRIFOLS, S.A.** cs=SCM_JShenSCM. pi=dbPool1 ui=new_user_id locale=en_US ktf1=[EM,EX,EH] if1=2950 ktf3=3x3 ktf4=TEAMVIEW ktf5=MYSELF ktf6=ALL ktf8=[gender,department,division,location] if2=0 if3=1 if4=0 > module=SCM fa=PP at=SCM.PP.GENERATE si=E02F07592081B6A58F7FFC3DE035D62B ci=JShenSCM **cn=Deutsche Telekom AG** cs=SCM_JShenSCM. pi=dbPool1 ui=new_user_id locale=en_US ktf1=[] if1=2950 ktf3=3x3 ktf4=TEAMVIEW ktf5=MYSELF ktf6=ALL ktf8=[gender,department,division,location] if2=0 if3=1 if4=0 > module=SCM fa=PP at=SCM.PP.GENERATE si=E02F07592081B6A58F7FFC3DE035D62B ci=JShenSCM **cn=Deutsche Bank** cs=SCM_JShenSCM. pi=dbPool1 ui=new_user_id locale=en_US ktf1=[EM,EX,EMM] if1=2950 ktf3=3x3 ktf4=TEAMVIEW ktf5=MYSELF ktf6=ALL ktf8=[] if2=0 if3=1 if4=0 > > ``` If field\_split\_patter =\> "[^,] " is applied like below, then cn=Deutsche Telekom AG OR cn=Deutsche Bank won't be parsed correctly, these two are all parsed to "cn=Deutsche" which is unexpected. > ``` > kv { > field_split => " " > field_split_pattern => "[^,] " > value_split => "=" > trim_key => " " > trim_value => " " > include_brackets => false } > > ``` Expected Result for cn field after parsing cn=GRIFOLS, S.A. cn=Deutsche Telekom AG cn=Deutsche Bank ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 18, 2019, 12:46pm UTC](https://discuss.elastic.co/t/es7-0-1-field-cn-keyword-text-type-is-not-extracted-as-expected-from-message-field/199560/4 "2019-09-18T12:46:10Z")
I don't see a way to fix that. You need to be able to tell the kv filter to use " " for field\_split _sometimes_, and there is no way to do that. ---
**Author:** ![cheriemilk](https://avatars.discourse-cdn.com/v4/letter/c/c37758/32.png) [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Post date:** [September 20, 2019, 5:51am UTC](https://discuss.elastic.co/t/es7-0-1-field-cn-keyword-text-type-is-not-extracted-as-expected-from-message-field/199560/5 "2019-09-20T05:51:34Z")
@Badger, As I mentioned in another topic , gsub might be a solution. But I can't make it work. I add below into logstash configuration ` > mutate { gsub =\> ["message", "(\s)\w+=", "?"] } ` and in regex101, it exactly matched the spaces that are the spliter of field. check here [regex101: build, test, and debug regex](https://regex101.com/r/ygsL8o/1) But when I configure it in logstash, I observe that the result is still unexpected. `module=SCM?PP?SCM.PP.GENERATE?E02F07592081B6A58F7FFC3DE035D62B?JShenSCM?Deutsche Telekom AG?SCM_JShenSCM.?dbPool1?new_user_id?en_US?[]?2950?3x3?TEAMVIEW?MYSELF?ALL?[gender,department,division,location]?0?1?0\r` It replace the whole regex pattern (\s)\w+= with '?', but shouldn't the \s in the parenthesis be replaced for gsub? Thanks, Cherie ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [September 20, 2019, 1:56pm UTC](https://discuss.elastic.co/t/es7-0-1-field-cn-keyword-text-type-is-not-extracted-as-expected-from-message-field/199560/6 "2019-09-20T13:56:35Z")
> [@cheriemilk](#): > > shouldn't the \s in the parenthesis be replaced for gsub? No. That's not the way capture groups work. You need to use a lookahead assertion, which I showed in my answer in the other thread. ---
**Author:** ![cheriemilk](https://avatars.discourse-cdn.com/v4/letter/c/c37758/32.png) [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Post date:** [September 26, 2019, 5:42am UTC](https://discuss.elastic.co/t/es7-0-1-field-cn-keyword-text-type-is-not-extracted-as-expected-from-message-field/199560/7 "2019-09-26T05:42:00Z")
@Badger Thanks for the solution ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 24, 2019, 5:42am UTC](https://discuss.elastic.co/t/es7-0-1-field-cn-keyword-text-type-is-not-extracted-as-expected-from-message-field/199560/8 "2019-10-24T05:42:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.