If in filter drops event

Hey all,

I am using CSV filter plugin and I want to check a field value and if exists I want to add a new field. However when I add an "if' condition , the relevant event is missing in the output

log event

<14>Dec 20 15:56:37 10.1.2.225 1,2019/12/20 15:56:37,001801033544,TRAFFIC,start,2049,2019/12/20 15:56:37,xxx.xxx.xxxx.xxx,,TFAPP02 access for Users,,,web-browsing,vsys1,10.0.0
.0-10.255.255.255,0,3,1,n/a,0,0,0,0,,from-policy,,,0,,0,,N/A,0,0,0,0

conf file

input {
file {
path => "/usr/src/plugin/paloalto/log.log"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
skip_empty_columns => true
columns => ["number","datetime","id","NETTYPE","action","col6", "col7" ,"src_ip","dst_ip"]
}
mutate { add_field => {"pluginid" => "90515" }}
mutate { add_field => {"datetype" => "1"}}
if [action] == "start" {
mutate { add_field => { "plugin_sid" => "456" }}
}
}
output {
stdout { codec => rubydebug }
}

I am getting all outputs except [action] = "start" events, if I remove the "if" condition I get all the events.

I get the following at the end of the output

{
"message" => "",
"path" => "/usr/src/plugin/paloalto/log.log",
"host" => "sm_siem_db_mw1",
"@timestamp" => 2019-12-22T02:24:54.268Z,
"pluginid" => "90515",
"@version" => "1",
"tags" => [
[0] "_csvparsefailure"
],
"datetype" => "1"
}

Hey All,
I found what the problem was with the events being dropped. Actually it was not getting dropped, but the output had put the "start" events to the very begging of the out put, so I was not able to see them.

However I still do not know what I get a "_csvparsefailure"

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.