Filebeat If Statement Ingest Processor not Working

Elastic Stack Beats
1

Hello,

I am trying to map an IP address to a hostname with an If statement on an ingest pipeline.
These IPs are not in DNS and need to have a hostname associated to them for reporting.

If I remove the "if" statement, the set processor works fine. I have tried variations on matching with If: and am not having any luck.

This is using the NetFlow module for Filebeats.

  "filebeat-7.5.2-netflow-log-pipeline" : {
    "description" : "Pipeline for Filebeat NetFlow",
    "processors" : [
      {
        "geoip" : {
          "if" : "ctx.source?.geo == null",
          "field" : "source.ip",
          "target_field" : "source.geo",
          "ignore_missing" : true
        }
      },
      {
        "geoip" : {
          "field" : "destination.ip",
          "target_field" : "destination.geo",
          "ignore_missing" : true,
          "if" : "ctx.destination?.geo == null"
        }
      },
      {
        "geoip" : {
          "database_file" : "GeoLite2-ASN.mmdb",
          "field" : "source.ip",
          "target_field" : "source.as",
          "properties" : [
            "asn",
            "organization_name"
          ],
          "ignore_missing" : true
        }
      },
      {
        "geoip" : {
          "properties" : [
            "asn",
            "organization_name"
          ],
          "ignore_missing" : true,
          "database_file" : "GeoLite2-ASN.mmdb",
          "field" : "destination.ip",
          "target_field" : "destination.as"
        }
      },
      {
        "rename" : {
          "field" : "source.as.asn",
          "target_field" : "source.as.number",
          "ignore_missing" : true
        }
      },
      {
        "rename" : {
          "ignore_missing" : true,
          "field" : "source.as.organization_name",
          "target_field" : "source.as.organization.name"
        }
      },
      {
        "rename" : {
          "target_field" : "destination.as.number",
          "ignore_missing" : true,
          "field" : "destination.as.asn"
        }
      },
      {
        "rename" : {
          "field" : "destination.as.organization_name",
          "target_field" : "destination.as.organization.name",
          "ignore_missing" : true
        }
      },
      {
        "set" : {
          "if" : "observer.ip == X.X.X.X",
          "field" : "observer.hostname",
          "value" : "HOSTNAME-HERE"
        }
      }
    ],
    "on_failure" : [
      {
        "set" : {
          "value" : "{{ _ingest.on_failure_message }}",
          "field" : "error.message"
        }
      }
    ]
  }

Thanks,

Jake

2

Hi Jake,

What is the error message you see when the condition is enabled?
Also, have you tried:

"if": "ctx.observer?.ip == 'x.x.x.x'",
3

Thank you Mariana!

I was having two issues. I was not using ctx correctly. Initially, I had ctx._source.observer.ip rather than ctx.observer?.ip

Second, I did not have the single quotes around the IP address. The final solution was like this:

  {
    "set" : {
      "if" : "ctx.observer?.ip == 'X.X.X.X'",
      "field" : "observer.hostname",
      "value" : "HOSTNAME-HERE"
    }
  }

Thanks again for your assistance.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.