Logstash DNS Filter - Enrichment security example

Elastic Stack Logstash
1

Hi All,

I was trying to execute the Data enrichment logstash -IP addresses exist in an external DNSBL (DNS Blocklist)?

filter {
   grok {
     match => { "message" => "%{WORD:addr1}.%{WORD:addr2}.%{WORD:addr3}.%{WORD:addr4}" }     <= extract different part of the IP address 
   }
   mutate {
     add_field => {
     "spamhaus_reverse_lookup" => "%{addr4}.%{addr3}.%{addr2}.%{addr1}.zen.spamhaus.org" . <= reformat the address for spamhaus query
     }
   }
   dns {
     resolve => [ "spamhaus_reverse_lookup" ] <= perform lookup using the reverse address format.
     nameserver => [ "x.x.x.x" ]
     add_tag => [ "dns_successful_lookup" ]
     action => replace
   }
   if "dns_successful_lookup" in [tags] {
     if [spamhaus_reverse_lookup] == "127.0.0.2" { <= 127.0.0.2 is a special return address from spamhaus that indicate the lookup address is listed in spamhaus spam database
       mutate {
          add_tag => [ "spam_address" ]
       }
     }
   }
}

I get a message stating
[FATAL] 2018-02-22 12:41:53.785 [LogStash::Runner] runner - The given configuration is invalid. Reason: Expected one of #, and, or, xor, nand, { at line 19, column 8 (byte 533) after filter {

But if I comment out the last section the filter it works fine ,am not sure whats the problem with that.

filter {
  if [type] == "spam" {
   grok {
     match => { "message" => "%{WORD:addr1}.%{WORD:addr2}.%{WORD:addr3}.%{WORD:addr4}" }
    }
   mutate {
     add_field => {
     "spamhaus_reverse_lookup" => "%{addr4}.%{addr3}.%{addr2}.%{addr1}.zen.spamhaus.org"
     }
   }
   dns {
     resolve => [ "spamhaus_reverse_lookup" ]
     nameserver => [ "x.x.x.x" ]
     add_tag => [ "dns_successful_lookup" ]
     action => replace
   }
#   if "dns_successful_lookup" in [tags] {
#     if [spamhaus_reverse_lookup] == "127.0.0.2"
#       mutate {
#          add_tag => [ "spam_address" ]
#        }
#    }
  }
}

Please do let me know  what could be the problem in the logstash filter.

Thanks,
Raj
2

Do you actually have the "<=" comments in your configuration file? Remove them, and watch out for the period at the end of one of the lines in the first mutate filter.

3

Thank you for the reply , actually I copied that configuration from that blog but my configuration looks like this

filter {
  if [type] == "spam" {
   grok {
     match => { "message" => "%{WORD:addr1}.%{WORD:addr2}.%{WORD:addr3}.%{WORD:addr4}" }
    }
   mutate {
     add_field => {
     "spamhaus_reverse_lookup" => "%{addr4}.%{addr3}.%{addr2}.%{addr1}.zen.spamhaus.org"
     }
   }
   dns {
     resolve => [ "spamhaus_reverse_lookup" ]
     nameserver => [ "x.x.x.x" ]
     add_tag => [ "dns_successful_lookup" ]
     action => replace
   }
   if "dns_successful_lookup" in [tags] {
     if [spamhaus_reverse_lookup] == "127.0.0.2"
       mutate {
          add_tag => "spam_address"
        }
    }
  }
}

Then i tried comment out the last section the filter like this

filter {
  if [type] == "spam" {
   grok {
     match => { "message" => "%{WORD:addr1}.%{WORD:addr2}.%{WORD:addr3}.%{WORD:addr4}" }
    }
   mutate {
     add_field => {
     "spamhaus_reverse_lookup" => "%{addr4}.%{addr3}.%{addr2}.%{addr1}.zen.spamhaus.org"
     }
   }
   dns {
     resolve => [ "spamhaus_reverse_lookup" ]
     nameserver => [ "x.x.x.x" ]
     add_tag => [ "dns_successful_lookup" ]
     action => replace
   }
#   if "dns_successful_lookup" in [tags] {
#     if [spamhaus_reverse_lookup] == "127.0.0.2"
#       mutate {
#          add_tag => [ "spam_address" ]
#        }
#    }
  }
}

it works fine.Please let me know what could be the problem
Thanks,
Raj

4

There's a { missing from the if [spamhaus_reverse_lookup] == "127.0.0.2" line.

This it stuff you can debug by commenting out pieces of the configuration until you narrow thing down to a single line or two.

5

Thank you for the reply