How can I pass a filebeat [host][ip] array to the logstash CIDR filter plugin?

Elastic Stack Logstash
1

I'm using the filebeat add_host_metadata processor to enrich events with an array of local IP addresses for a host but I can't pass that to the logstash CIDR filter plugin because it sees it as a string, not as an array of strings.

Filebeat add host metadata: https://www.elastic.co/guide/en/beats/filebeat/master/add-host-metadata.html

Logstash CIDR Filter Plugin:
https://www.elastic.co/guide/en/logstash/current/plugins-filters-cidr.html

FILEBEAT CONFIGURATION

processors:
- add_host_metadata:
    netinfo.enabled: true

Filebeat passes the following to logstash

"host": {
      "ip": [
        "fe80::xxxx:xxxx:xxxx:xxx",
        "10.x.x.x",
        "fe80::xxxx:xxxx:xxxx:xxx",
        "10.x.x.x"
      ],
      "name": "HOSTNAME"
    },

I want to pass the [host][ip] array to the logstash CIDR filter plugin below.
LOGSTASH CONFIGURATION

input {
  beats { 
    port => 5044
  }
}
filter {
    cidr {
      add_field => { "[corp][environment]" => "prod.corp.com" }
      address => "%{[host][ip]}"
      network => [ "10.x.0.0/16" ]
    }
}

However, when I do, I get the following error because the array isn't getting passed as an array:

[WARN ][logstash.filters.cidr    ] Invalid IP address, skipping {:address=>"%{[host][ip]}", :event=>#<LogStash::Event:0x61279c17>}

It does work if I pass each individually but that is extremely messy as I don't know how many values there are for each device:

input {
  beats { 
    port => 5044
  }
}
cidr {
  add_field => { "[corp][environment]" => "prod.corp.com" }
  address => ["%{[host][ip][0]}", "%{[host][ip][1]}","%{[host][ip][2]}","%{[host][ip][3]}","%{[host][ip][4]}"]
  network => [ "10.x.0.0/16" ]
}

}

This works but is messy

[DEBUG][logstash.filters.cidr    ] Checking IP inclusion {:address=>#<IPAddr: IPv6:fe80:0000:0000:0000:xxxx:xxxx:xxxx:xxxx/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff>, :network=>#<IPAddr: IPv4:10.x.x.0/255.255.0.0>}
[DEBUG][logstash.filters.cidr    ] Checking IP inclusion {:address=>#<IPAddr: IPv4:10.x.x.x/255.255.255.255>, :network=>#<IPAddr: IPv4:10.x.0.0/255.255.0.0>}
[DEBUG][logstash.filters.cidr    ] Checking IP inclusion {:address=>#<IPAddr: IPv6:fe80:0000:0000:0000:xxxx:xxxx:xxxx:xxxx/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff>, :network=>#<IPAddr: IPv4:10.x.x.0/255.255.0.0>}
[DEBUG][logstash.filters.cidr    ] Checking IP inclusion {:address=>#<IPAddr: IPv4:10.x.x.x/255.255.255.255>, :network=>#<IPAddr: IPv4:10.x.0.0/255.255.0.0>}
[WARN ][logstash.filters.cidr    ] Invalid IP address, skipping {:address=>"%{[host][ip][4]}", :event=>#<LogStash::Event:0x78d60b5f>}

Is this a general logstash issue or is this a bug with the CIDR filter plugin?

Thanks!

Peter

2

The CIDR filter plugin appears to not be designed to work with fields that contain array values, because it uses an array of sprintf templates for its input.

I've filed an issue on the project repository: https://github.com/logstash-plugins/logstash-filter-cidr/issues/19

3

Thanks very much.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.