Tagging for public IP address

[psmaan1]

Hi,
I am ingesting windows event logs in ELK setup (7.0.1). I need to add a tag to events e.g. "src_public" or "dest_public" whenever source IP or Destination IP or both in the logs are public IP address.
For windows Logon event, IP is recorded in winlog.event_data.IpAddress.
For firewall events, IP is recorded in winlog.event_data.SourceAddress and winlog.event_data.DestAddress.
*Only If I could negate the selection in cidr filter, it would be super easy:

filter {
if [winlog][event_data][IpAddress] {
cidr{
add_tag => [ "src_public"]
address => [ "%{[winlog][event_data][IpAddress]}" ]
network => NOT [ "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16" ] }
} }

But I don`t think this is possible.
Is there any way to do it otherwise.

I am going to use this tag in my geo IP filter, where I would check if tag is public to resolve the Lat/Long fields.

[Badger]

Rather than negating the selection in the filter, negate the conditional for the resulting tag being present. Or if you absolutely need a tag to indicate a private IP address, use mutate+add_tag inside a conditional that excludes events with tag src_public.

[stcdarrell (Darrell Miller)]

like badget suggested:
filter {
if [winlog][event_data][IpAddress] {
cidr{
add_tag => [ "src_private"]
address => [ "%{[winlog][event_data][IpAddress]}" ]
network => [ "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16" ] }
}#end cidr
if "src_private" not in [tags] {
mutate {
add_tag => "src_public"
} #end mutate
}#end if
}#end filter

[psmaan1]

I tried it. It will introduce another problem : any logs which will not have this tag, will get alternate tag. To solve that, I may have to check the condition that IP fields are present in the logs to limit tag application to only those logs where it is required.

Secondly, There are addresses which does not fall under private/public IP ranges as well.
I guess for 127.0.0.1, I would need to use something like
if [winlog][event_data][IpAddress] == "127.0.0.1", as I suppose cidr does not support single IP address.
For cases where, IP is blank, or ::1 or a IPv6, I need to put similar conditions.
Is it possible to have case statements? Do you think that`s a viable option here.

[Badger]

@psmaan1

suppose cidr does not support single IP address.

The cidr filter will support 127.0.0.1/32 as a network, however, you probably want 127.0.0.0/8. I routinely use other addresses on the loopback adapter.

[psmaan1]

I was able to solve this issue using Regex on fields rather than CIDR.
Here is my solution:

filter {
if [winlog][event_data][IpAddress] !~ /^(10|127|169.254|172.1[6-9]|172.2[0-9]|172.3[0-1]|192.168)./ {
if [winlog][event_data][IpAddress] =~ /^\b(?:\d{1,3}.){3}\d{1,3}\b/ {
if [winlog][event_data][IpAddress] !~ /^(0.0.0.0)./ {
mutate { add_tag => "public_ip" }
} # end 3rd if
} # end 2nd if
} # end 1st if
} # end filter

I need to use last filter for "0.0.0.0" it is passing all the checks before.

[Badger]

I realize that that may solve your current problem, but in the long run you would probably be better off using a cidr filter to do CIDR pattern matching.

[psmaan1]

Hey Badger,
Following your advice, I tried to do it using cidr. I end up combining both approaches.
Here is how the solution looks like now:

filter {
if [winlog][task] == "Logon" {
cidr{
add_tag => [ "src_private"]
address => [ "%{[winlog][event_data][IpAddress]}" ]
network => [ "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "127.0.0.0/8" ] }

if [winlog][event_data][IpAddress] =~ /^\b(?:\d{1,3}.){3}\d{1,3}\b/ and "src_private" not in [tags] {
if [winlog][event_data][IpAddress] !~ /^(0.0.0.0)/ {
mutate { add_tag => "src_public" }
} }
}
}

However, I am getting lot of " Invalid IP Address, skipping **" logs in logstash log file from cidr. I believe these are those entries which does not have a valid IP e.g. Null IP, ::1 etc.

Is there any way I can get rid of these, as this would fill up my logs files and disk space.

[Badger]

The cidr filter is able to parse ::1. If the address is not parseable then the filter will unconditionally log a warning. You cannot prevent it.