# Duplicate entries in Logstash?

**URL:** <https://discuss.elastic.co/t/duplicate-entries-in-logstash/33571>\
**Category:** Logstash\
**Created:** [November 2, 2015, 9:32pm UTC](https://discuss.elastic.co/t/duplicate-entries-in-logstash/33571 "2015-11-02T21:32:04Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kmroz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kmroz/32/62741_2.png) [@kmroz](https://discuss.elastic.co/u/kmroz)\
**Post date:** [November 2, 2015, 9:32pm UTC](https://discuss.elastic.co/t/duplicate-entries-in-logstash/33571/1 "2015-11-02T21:32:04Z")

</div>

I am not to sure where the issue is if it is with Logstash or Log-courier. It seems that logstash keeps rereading the same log line and parses it multiple times. To check this i named the common field type %{WORD:field1},%{WORD:field2},%{WORD:field3},. When i tail the log for logstash each one come out thress different times for that same log.  
"@version" =\> "1",  
"@timestamp" =\> "2015-11-02T21:25:34.495Z",  
"day" =\> "Mon",  
"timestamp" =\> "Nov 02 21:25:30",  
"year" =\> "2015",  
"global1" =\> "REX", (This repeats itself three time, same time stamp and log entry)  
"resource" =\> "Service",  
"status" =\> "notice",  
"results" =\> " enable changed 'false' to 'true'"

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 3, 2015, 7:02am UTC](https://discuss.elastic.co/t/duplicate-entries-in-logstash/33571/2 "2015-11-03T07:02:11Z")

</div>

What version?  
What do your configs look like?

---

<div class="post-metadata">

**Author:** ![kmroz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kmroz/32/62741_2.png) [@kmroz](https://discuss.elastic.co/u/kmroz)\
**Post date:** [November 3, 2015, 2:46pm UTC](https://discuss.elastic.co/t/duplicate-entries-in-logstash/33571/3 "2015-11-03T14:46:35Z")

</div>

logstash=1.5 log-courier=latest

input {  
courier {  
port =\> 5001  
type =\> "logs"  
ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt"  
ssl\_key =\> "/etc/pki/tls/private/logstash-forwarder.key"  
}

}  
filter {  
if [type] == "puppet" {  
grok {  
match =\> { "message"=\>"%{DAY:day}\ %{SYSLOGTIMESTAMP:timestamp}\ %{INT:notused}\ %{YEAR:year}\ /Stage[main]/%{WORD:global1}::%{NOTSPACE:nothing}/%{WORD:resource}[%{DATA:name}]/%{WORD:nothing} (%{LOGLEVEL:status}):%{GREEDYDATA:results}"}  
match =\> { "message"=\>"%{DAY:day}\ %{SYSLOGTIMESTAMP:timestamp}\ %{INT:notused}\ %{YEAR:year}\ /Stage[main]/%{WORD:global2}/%{WORD:resource}[%{DATA:name}]/%{NOTSPACE:nothing}\ (%{LOGLEVEL:status}):%{GREEDYDATA:results}"}  
match =\> { "message"=\>"%{DAY:day}\ %{SYSLOGTIMESTAMP:timestamp}\ %{INT:notused}\ %{YEAR:year}\ %{DATA:application}]/%{WORD:global3}%{DATA:nothing}[%{DATA:name}] (%{LOGLEVEL:status}):\ %{GREEDYDATA:results}"}  
match =\> { "message"=\>"%{DAY:day}\ %{SYSLOGTIMESTAMP:timestamp}\ %{INT:notused}\ %{YEAR:year}\ %{DATA:application}\ (%{LOGLEVEL:status}):\ %{GREEDYDATA:results}"}

```
    }

                    }

```

mutate{  
remove\_field =\> ["predata", "number","nothing","application"]  
remove\_field =\> ["notused","stage","define"]  
remove\_field =\> ["md5", "main","words"]  
}  
}

output {  
elasticsearch { host =\> localhost }

}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:24am UTC](https://discuss.elastic.co/t/duplicate-entries-in-logstash/33571/4 "2017-07-06T05:24:17Z")

</div>