Create new columns from one column value


(Varun Goyal) #1

I have a csv columns as:

Data1,Data2,"Product: XYZ\r\nVersion: 1.0.0\r\nLanguage: English\r\nDate: Wednesday, September 12, 2018\r\n"

By using filter I am able to get data in columns, here is my filter:

csv {
#autodetect_column_names => true
columns => ["F1","F2","F3"]
source => "message"

}
mutate {
add_field => { "NEW_FIELD" => "%{F1}" }
}

I want to split F3 Data which is:
"Product: XYZ\r\nVersion: 1.0.0\r\nLanguage: English\r\nDate: Wednesday, September 12, 2018\r\n"
in key value pair.

Can any one help how can i achieve this ?


(Ry Biesemeyer) #2

First, you may want to use the Mutate Filter Plugin's copy directive, which is a little nicer about copying edge-case values than add_field with string interpolation.

mutate {
  copy => {
    "F1" => "NEW_FIELD"
  }
}

Next, You could likely use the KV Filter Plugin to split out the key/value pairs from the F3 field:

kv {
  source => "F3"
  field_split => "\n"
  value_split => ":"
  trim_value => "\r\s" # trim carriage-returns and whitespace from beginning and end of value 
}

(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.