# Duplicate data showing in interesting fields

**URL:** <https://discuss.elastic.co/t/duplicate-data-showing-in-interesting-fields/105831>\
**Category:** Logstash\
**Created:** [October 31, 2017, 3:14am UTC](https://discuss.elastic.co/t/duplicate-data-showing-in-interesting-fields/105831 "2017-10-31T03:14:35Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Feedy](https://avatars.discourse-cdn.com/v4/letter/f/76d3ee/32.png) [@Feedy](https://discuss.elastic.co/u/Feedy)\
**Post date:** [October 31, 2017, 3:14am UTC](https://discuss.elastic.co/t/duplicate-data-showing-in-interesting-fields/105831/1 "2017-10-31T03:14:35Z")

</div>

Can someone take a look at my configs to help me understand why I am getting duplicate data for some parsed fields? See screenshot of Kibana showing duplicate fields and snort-filter.conf on logstash server:  
 ![Capture](https://us1.discourse-cdn.com/elastic/original/3X/0/f/0faf156f6b6cf4bbbe1dce1dfecd9b0d019f15e3.PNG)

snort-filter.conf:  
filter {  
if [type] == "snort"{  
grok {  
break\_on\_match =\> false  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:Host} [%{INT:ids\_gid}:%{INT:ids\_sid}:%{INT:ids\_rev}] (%{DATA:preprocessor})?: %{GREEDYDATA:message} [Classification: %{DATA:ids\_classification}] [Priority: %{INT:priority}]: ({%{WORD:ids\_protocol}}) %{IP:src\_ip}:%{INT:src\_port} -\> %{IP:dst\_ip}:%{INT:dst\_port}"}  
}  
grok {  
break\_on\_match =\> false  
match =\> { "message" =\> "%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:Host} [%{INT:ids\_gid}:%{INT:ids\_sid}:%{INT:ids\_rev}] %{GREEDYDATA:message} [Classification: %{DATA:ids\_classification}] [Priority: %{INT:priority}]: ({%{WORD:ids\_protocol}}) %{IP:src\_ip}:%{INT:src\_port} -\> %{IP:dst\_ip}:%{INT:dst\_port}"}  
}  
}  
geoip {  
source =\> "[src\_ip]"  
target =\> "SrcGeo"  
}  
geoip {  
source =\> "[dst\_ip]"  
target =\> "DstGeo"  
}  
if [priority] == "1" {  
mutate {  
add\_field =\> { "severity" =\> "High" }  
}  
}  
if [priority] == "2" {  
mutate {  
add\_field =\> { "severity" =\> "Medium" }  
}  
}  
if [priority] == "3" {  
mutate {  
add\_field =\> { "severity" =\> "Low" }  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 31, 2017, 8:27pm UTC](https://discuss.elastic.co/t/duplicate-data-showing-in-interesting-fields/105831/2 "2017-10-31T20:27:32Z")

</div>

You're extracting the same fields from both grok filters. Either use a single expression (your current expressions are almost identical and should be easy to get into a single expression) or use two expressions in a single grok filter (the filter docs contain an example of that).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 28, 2017, 8:30pm UTC](https://discuss.elastic.co/t/duplicate-data-showing-in-interesting-fields/105831/3 "2017-11-28T20:30:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.