When I use the de_dot filter I am using a conditional statement that states:
filter { if [type] == "BRO" { if [log_path] == "software" { de_dot { fields => [ "version.major", "version.minor", "version.minor2", "version.minor3", "version.addl" ] } } if [log_path] == "x509" { de_dot { fields => [ "certificate.version", "certificate.serial", "certificate.subject", "certificate.issuer", "certificate.exponent", "certificate.curve", "sans.dns", "basic_constraints.ca" ] } } if [log_path] == "intel" { de_dot { fields => [ "seen.indicator", "seen.where", "seen.node" ] } } [rest of filters] }
but this has the unintended consequence of adding the de_dot'd field to the record even if it doesn't have that field. My issue is that sometimes these logs contain the fields and sometimes they don't. But the de_dot filter is adding the field (without the dots) to every record creating nil fields, see below:
"version_minor2" => nil, "version_minor3" => nil, "version_addl" => nil
Is there a way to tell the filter not to add the field. I stumbled upon this issue because I initially tried to use the syntax:
if [log_path] in [ "software","intel","x509" ] { de_dot { fields => [ .......
But what this did is append all records in the above ref'd logs with all de_dot'd fields creating a very large amount of nil fields.
So back to the question.....Is there a way to have de_dot look for the field specified and only de_dot it if it exists, or should I do some @metadata tagging to say if that field exists tag with a new metadata field then apply de_dot if that condition is met?