# How to Split the one field into multiple fields using kv plugins **URL:** \ **Category:** Logstash\ **Created:** [March 13, 2019, 6:41am UTC](https://discuss.elastic.co/t/how-to-split-the-one-field-into-multiple-fields-using-kv-plugins/172067 "2019-03-13T06:41:04Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![muralasandeep](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@muralasandeep](https://discuss.elastic.co/u/muralasandeep)\ **Post date:** [March 13, 2019, 6:41am UTC](https://discuss.elastic.co/t/how-to-split-the-one-field-into-multiple-fields-using-kv-plugins/172067/1 "2019-03-13T06:41:04Z")
Hi , This is Sandeep, I am new at logstash, here i am trying to fatch data from .csv file to elasticsearch using logstash and i have 6 fields total , in that 6 fields one of the field(log\_message) have actually data i need to split that field as multiple fields , here is my config file ```auto input{ file{ path => "/home/Data/logs.csv" start_position => "beginning" sincedb_path => "/dev/null" } } filter{ csv{ separator => "," columns => ["log_id","log_datetime","log_type","log_message","log_user"] } date { match => ["log_datetime", "yyyy-MM-dd HH:mm:ss"] } mutate { convert => ["log_id","integer"] } kv { field_split_pattern => "[\"]+" value_split_pattern => "->" } kv { field_split_pattern => "[\( \)]+" value_split_pattern => "->" } kv { field_split_pattern => "[]+" value_split_pattern => ":" } } output{ elasticsearch{ hosts =>"http://localhost:9200" index =>"log_infomation" document_type => "log_info" } stdout { codec => "rubydebug" } } ``` I am getting the data when i use kv { field\_split\_pattern =\> "["]+" value\_split\_pattern =\> "-\>" } kv { field\_split\_pattern =\> "[( )]+" value\_split\_pattern =\> "-\>" } kv { field\_split\_pattern =\> "+" value\_split\_pattern =\> ":" } but in my log\_message field some rows are like this (assume 1st row) "(Institution Id-\>77) (data Id-\>127) QUERY-\> insert into tablename1(col1,col2) values(val1,val2) ERROR-\>Table 'tablename1' doesn't exist" some are (assume 2st row) "QUERY-\> update set tablename2 col1=val1 some conditions ERROR: Table 'Tablename2' doesn't exist" and some rows are like this (assume 3rd row) "QUERY-\> update set tablename3 col1=val1 some conditions ERROR: You have SQL syntax error " if i apply "kv" filter plugins i am getting the results like this , 1st document like this from the first row data, { "ERROR" : "Table", //here i am getting only first word from log\_message field "QUERY" : "insert", //here also "Id" :[ "77", "127" ] }, 2nd document like this, { "ERROR" : "Table", //here i am getting only first word "QUERY" : "update", //here also }, 3rd document like this { "Error" : "update", "Query" : "You" } but here my expected result is { "ERROR" : "Table 'tablename1' doesn't exist", "QUERY" : "insert into tablename1(col1,col2) values(val1,val2) ", //i need total query here not only insert command "institution Id" : 77, //here i need field name like "institution Id" "data Id" : 127 //here i need field name like "data Id" }, { "ERROR" : "Table 'tablename2' doesn't exist", "QUERY" : "update set tablename2 col1=val1 some conditions", //i need total query here not only insert command "institution Id" : 77, //here i need field name like "institution Id" "data Id" : 127 //here i need field name like "data Id" }, { "Query": "update set tablename3 col1=val1 some conditions", "Error": " You have SQL syntax error" } OR { "QUERY" : "insert into tablename1(col1,col2) values(val1,val2) "ERROR" =\> "Table 'tablename1' doesn't exist" "ERROR" : "Table 'tablename1' doesn't exist", "institution Id" : 77, //here i need field name like "institution Id" "data Id" : 127 //here i need field name like "data Id" }, { "QUERY" : "update set tablename2 col1=val1 some conditions "ERROR" =\> "Table 'tablename2' doesn't exist" "ERROR" : "Table 'tablename2' doesn't exist", "institution Id" : 77, //here i need field name like "institution Id" "data Id" : 127 //here i need field name like "data Id" }, { "Query": "update set tablename3 col1=val1 some conditions Error: You have SQL syntax error", "Error": " You have SQL syntax error" } what should i do in this case ,can any one help me please ? Thanks Sandeep ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 13, 2019, 10:50am UTC](https://discuss.elastic.co/t/how-to-split-the-one-field-into-multiple-fields-using-kv-plugins/172067/2 "2019-03-13T10:50:50Z")
I don't think your data is regular enough to use a kv plugin. grok might work. ---
**Author:** ![muralasandeep](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@muralasandeep](https://discuss.elastic.co/u/muralasandeep)\ **Post date:** [March 13, 2019, 11:36am UTC](https://discuss.elastic.co/t/how-to-split-the-one-field-into-multiple-fields-using-kv-plugins/172067/3 "2019-03-13T11:36:57Z")
Hi @Badger Thank you for reply, i have less knowledge on grok filter because i am new at logstash so, can i do this grok filter only particular field like log\_message because i need only one field data . columns are like this , columns =\> ["log\_id","log\_datetime","log\_type","log\_message","log\_user"] this log\_message data shown in above context . please help me , Thanks Sandeep ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 13, 2019, 12:53pm UTC](https://discuss.elastic.co/t/how-to-split-the-one-field-into-multiple-fields-using-kv-plugins/172067/4 "2019-03-13T12:53:00Z")
If an example of log\_message is ```auto (Institution Id->77) (data Id->127) QUERY-> insert into tablename1(col1,col2) values(val1,val2) ERROR->Table 'tablename1' doesn't exist ``` Then you could parse that (and the other 2) using this ``` grok { match => { "log_message" => "^%{DATA:prefix}QUERY\-> %{DATA:query} ERROR(: |\->)%{DATA:error}$" } } if [prefix] { ruby { code => ' m = event.get("prefix").scan(/\(([^-]+)\->([^\)]+)\)/) m.each.each { |k, v| event.set(k, v) } ' } } ``` In the ruby filter the scan function matches a regexp against the "prefix" field. The regexp looks for (key-\>value) patterns and returns an array of them. Each match is on entry in the array, and that entry itself is an array, the first entry of which is the key, and the second is the value. ---
**Author:** ![muralasandeep](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@muralasandeep](https://discuss.elastic.co/u/muralasandeep)\ **Post date:** [March 14, 2019, 5:17am UTC](https://discuss.elastic.co/t/how-to-split-the-one-field-into-multiple-fields-using-kv-plugins/172067/5 "2019-03-14T05:17:55Z")
Hi @Badger , Thank you so much for replaying, It is working perfectly ,i really appreciate you thank you so much . Thanks Sandeep ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 11, 2019, 5:18am UTC](https://discuss.elastic.co/t/how-to-split-the-one-field-into-multiple-fields-using-kv-plugins/172067/6 "2019-04-11T05:18:10Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.