De_dot adding fields

Elastic Stack Logstash
1

When I use the de_dot filter I am using a conditional statement that states:

filter {
  if [type] == "BRO" {
    if [log_path] == "software" {
      de_dot {
        fields => [
          "version.major",
          "version.minor",
          "version.minor2",
          "version.minor3",
          "version.addl"
        ]
      }
    }
    if [log_path] == "x509" {
      de_dot {
        fields => [
          "certificate.version",
          "certificate.serial",
          "certificate.subject",
          "certificate.issuer",
          "certificate.exponent",
          "certificate.curve",
          "sans.dns",
          "basic_constraints.ca"
        ]
      }
    }
    if [log_path] == "intel" {
      de_dot {
        fields => [
          "seen.indicator",
          "seen.where",
          "seen.node"
        ]
      }
    }
[rest of filters]
}

but this has the unintended consequence of adding the de_dot'd field to the record even if it doesn't have that field. My issue is that sometimes these logs contain the fields and sometimes they don't. But the de_dot filter is adding the field (without the dots) to every record creating nil fields, see below:

      "version_minor2" => nil,
      "version_minor3" => nil,
        "version_addl" => nil

Is there a way to tell the filter not to add the field. I stumbled upon this issue because I initially tried to use the syntax:

    if [log_path] in [ "software","intel","x509" ] {
      de_dot {
        fields => [
         .......

But what this did is append all records in the above ref'd logs with all de_dot'd fields creating a very large amount of nil fields.

So back to the question.....Is there a way to have de_dot look for the field specified and only de_dot it if it exists, or should I do some @metadata tagging to say if that field exists tag with a new metadata field then apply de_dot if that condition is met?

2

Please raise this as an issue at https://github.com/logstash-plugins/logstash-filter-de_dot

We can add functionality to check if the field exists before acting. Until we do, your remedy is to use conditionals and tag, or to let it look at all fields (or use a combination to only look at events which may have dotted fields).

3

Thank you