Duplicate data showing in interesting fields

Elastic Stack Logstash
1

Can someone take a look at my configs to help me understand why I am getting duplicate data for some parsed fields? See screenshot of Kibana showing duplicate fields and snort-filter.conf on logstash server:
Capture

snort-filter.conf:
filter {
if [type] == "snort"{
grok {
break_on_match => false
match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:Host} [%{INT:ids_gid}:%{INT:ids_sid}:%{INT:ids_rev}] (%{DATA:preprocessor})?: %{GREEDYDATA:message} [Classification: %{DATA:ids_classification}] [Priority: %{INT:priority}]: ({%{WORD:ids_protocol}}) %{IP:src_ip}:%{INT:src_port} -> %{IP:dst_ip}:%{INT:dst_port}"}
}
grok {
break_on_match => false
match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:Host} [%{INT:ids_gid}:%{INT:ids_sid}:%{INT:ids_rev}] %{GREEDYDATA:message} [Classification: %{DATA:ids_classification}] [Priority: %{INT:priority}]: ({%{WORD:ids_protocol}}) %{IP:src_ip}:%{INT:src_port} -> %{IP:dst_ip}:%{INT:dst_port}"}
}
}
geoip {
source => "[src_ip]"
target => "SrcGeo"
}
geoip {
source => "[dst_ip]"
target => "DstGeo"
}
if [priority] == "1" {
mutate {
add_field => { "severity" => "High" }
}
}
if [priority] == "2" {
mutate {
add_field => { "severity" => "Medium" }
}
}
if [priority] == "3" {
mutate {
add_field => { "severity" => "Low" }
}
}
}

2

You're extracting the same fields from both grok filters. Either use a single expression (your current expressions are almost identical and should be easy to get into a single expression) or use two expressions in a single grok filter (the filter docs contain an example of that).

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.