Hi guys, I still rely on an old minemeld installation that exports its IOC to and Elasticsearch index. Exported ip ranges are similar to this one: 104.156.155.0-104.156.155.255, that is not a CIDR notation nor and elastic proper range, and I need to look for matchings between packetbeat data and this ones.
So I have set up the appended logstash pipeline based on elasticsearch filter and tried with every query I could think: type:IPv4 AND @indicator: x.x.x.*, type:IPv4 AND @indicator: [x.x.x.0 TO x.x.x.255], etc but everytime the logstash filter append unwanted double quotes, square brackets or simply doesn't read the value from the message field.
With the last pipeline the error is the following: Cannot parse 'type:IPv4 AND @indicator: [%{search_lookup}.0 TO %{search_lookup}.0]'
Any suggestions?
input {
elasticsearch {
hosts => ["elastic1.local", "elastic2.local"]
index => "packetbeat-7.17.21*"
size => 10000
schedule => "*/5 * * * *"
query => '{
"query": {
"query_string": {
"query": "_exists_: dns.answers AND NOT event.type: end AND dns.response_code: NOERROR AND @timestamp: [now-5m TO now]",
"analyze_wildcard": true,
"time_zone": "Europe/Rome"
}
}
}'
}
}
filter {
dissect {
mapping => { "[dns][resolved_ip]" => "%{search_lookup}.%{+search_lookup}.%{+search_lookup}.%{}" }
}
mutate {
convert => { search_lookup => "string" }
}
mutate {
add_field => {
search_ip => "%{[search_lookup]}.0 TO %{[search_lookup]}.0"
}
}
mutate {
convert => { search_ip => "string" }
}
mutate {
gsub => [ "search_ip", "[\[\]\"]", "" ]
}
mutate {
gsub => [ "search_ip", "[\\]", "" ]
}
elasticsearch {
hosts => ["elastic1.local", "elastic2.local"]
index => "minemeld-*"
query => "type:IPv4 AND @indicator: [%{search_ip}]"
fields => { "@indicator" => "malware"}
}
if !([malware]) {
drop{}
}
}
output {
if !("_elasticsearch_lookup_failure" in [tags]) {
elasticsearch {
hosts => ["elastic1.local", "elastic2.local"]
index => [ "malwarefound-%{+YYYY.MM.dd}" ]
}
}
}