Hey all,
I am using CSV filter plugin and I want to check a field value and if exists I want to add a new field. However when I add an "if' condition , the relevant event is missing in the output
log event
<14>Dec 20 15:56:37 10.1.2.225 1,2019/12/20 15:56:37,001801033544,TRAFFIC,start,2049,2019/12/20 15:56:37,xxx.xxx.xxxx.xxx,,TFAPP02 access for Users,,,web-browsing,vsys1,10.0.0
.0-10.255.255.255,0,3,1,n/a,0,0,0,0,,from-policy,,,0,,0,,N/A,0,0,0,0
conf file
input {
file {
path => "/usr/src/plugin/paloalto/log.log"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
skip_empty_columns => true
columns => ["number","datetime","id","NETTYPE","action","col6", "col7" ,"src_ip","dst_ip"]
}
mutate { add_field => {"pluginid" => "90515" }}
mutate { add_field => {"datetype" => "1"}}
if [action] == "start" {
mutate { add_field => { "plugin_sid" => "456" }}
}
}
output {
stdout { codec => rubydebug }
}
I am getting all outputs except [action] = "start" events, if I remove the "if" condition I get all the events.
I get the following at the end of the output
{
"message" => "",
"path" => "/usr/src/plugin/paloalto/log.log",
"host" => "sm_siem_db_mw1",
"@timestamp" => 2019-12-22T02:24:54.268Z,
"pluginid" => "90515",
"@version" => "1",
"tags" => [
[0] "_csvparsefailure"
],
"datetype" => "1"
}