# Logstash Grok - duplicate entries after parsing
**URL:** https://discuss.elastic.co/t/logstash-grok-duplicate-entries-after-parsing/120563
**Category:** Logstash
**Created:** [February 20, 2018, 6:27am UTC](https://discuss.elastic.co/t/logstash-grok-duplicate-entries-after-parsing/120563 "2018-02-20T06:27:45Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@ranganath\_nangineni](https://discuss.elastic.co/u/ranganath_nangineni)
#### Post date: [February 20, 2018, 6:27am UTC](https://discuss.elastic.co/t/logstash-grok-duplicate-entries-after-parsing/120563/1 "2018-02-20T06:27:46Z")
I have a peculiar situation where , the grok match parsing is putting duplicate entries after parsing.
I have tried break\_on\_match =\> true ( with true -\> it is not able to parse the required fields), on false, it is putting duplicate entries.
The below is the grok pattern used"
# Logstash Grok pattern
if [fields][app] == "xyz" {
```
grok {
patterns_dir => ["/usr/share/logstash/patterns"]
match => {"message" => "%{COMBINEDAPACHELOG} %{QS:trueclientip} %{QS:filetype}"}
match => { "message" => "%{COMBINEDAPACHELOG} \"%{IP:trueclientip}\" %{QS:filetype}" }
break_on_match => false
}
```
# JSON output
{
"\_index": "XYZ-prod-cons-adc-ohs-2018.02.19",
"\_type": "ohs",
"\_id": "AWGu1kCJ\_An67UGq4G4Q",
"\_version": 1,
"\_score": null,
"\_source": {
"request": [
"/content/web/HRPAGE123",
"/content/web/HRPAGE123"
],
"filetype": [
""image/jpeg"",
""image/jpeg""
],
"agent": [
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko",
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
],
"client\_timezone": " North America Central time(CT)",
"auth": [
"john.k@xyz.com",
"john.k@xyz.com"
],
"ident": [
"-",
"-"
],
"tz": [
"-0600",
"-0600"
],
"trueclientip": [
"100.17.21.74",
"100.17.21.74"
],
"cl-geoip": {
"timezone": "America/Los\_Angeles",
"continent\_code": "NA",
"city\_name": "Redwood City",
"country\_name": "United States",
"country\_code2": "US",
"dma\_code": 807,
"country\_code3": "US",
"region\_name": "California",
"postal\_code": "94065",
"region\_code": "CA"
},
"tcl-geoip": {},
"clientip": [
"10.17.21.74",
"10.17.21.74"
],
"@version": "1",
"host": "XYZserver1",
"referrer\_host": [
"[XYZ.com](http://XYZ.com)",
"[XYZ.com](http://XYZ.com)"
],
"verb": [
"GET",
"GET"
],
"message": "10.17.21.74 - - [15/Feb/2018:04:28:59 -0600] "GET /content/web/HRPAGE123 HTTP/1.1" 200 28085 "[https://XYZ.com/site/fin/gfo/GlobalProcesses/GPTD/XyZ4358.html](https://XYZ.com/site/fin/gfo/GlobalProcesses/GPTD/XyZ4358.html)" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" "100.17.21.74" "image/jpeg" ",
"tags": [
"beats\_input\_codec\_plain\_applied",
"\_grokparsefailure",
"\_dateparsefailure",
"\_geoip\_lookup\_failure"
],
"referrer": [
"[https://XYZ.com/site/fin/gfo/GlobalProcesses/GPTD/XyZ4358.html](https://XYZ.com/site/fin/gfo/GlobalProcesses/GPTD/XyZ4358.html)",
"[https://XYZ.com/site/fin/gfo/GlobalProcesses/GPTD/XyZ4358.html](https://XYZ.com/site/fin/gfo/GlobalProcesses/GPTD/XyZ4358.html)"
],
"@timestamp": "2018-02-19T15:43:54.712Z",
"filename": [
"HRPAGE123",
"HRPAGE123"
],
"response": [
"200",
"200"
],
"bytes": [
"28085",
"28085"
],
"httpversion": [
"1.1",
"1.1"
],
"fields": {
"app": "XYZ",
"log\_type": "ohs",
"tier": "cons",
"lc": "prod",
"property": "[XYZ.com](http://XYZ.com)",
"dc": "adc"
},
"lob": [
"fin",
"fin"
]
},
"fields": {
"@timestamp": [
1519055034712
]
},
"sort": [
1519055034712
]
}
Please let me know , why is this happening?
How can I avoid this situation?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [February 20, 2018, 8:45am UTC](https://discuss.elastic.co/t/logstash-grok-duplicate-entries-after-parsing/120563/2 "2018-02-20T08:45:55Z")
I suspect this is because you have not configured this as an [array of index patterns](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#plugins-filters-grok-match) as described in the docs. As the first pattern seems to match anything the second one does, you might actually be able to simply remove the second pattern.
---
### Author:  [@ranganath\_nangineni](https://discuss.elastic.co/u/ranganath_nangineni)
#### Post date: [February 20, 2018, 10:32am UTC](https://discuss.elastic.co/t/logstash-grok-duplicate-entries-after-parsing/120563/3 "2018-02-20T10:32:35Z")
Both the patterns are the same.
When I have the first pattern , it is not getting parsed, but when I put both the patterns , both are getting parsed against the same log format record and putting 2 entries for each field. Which is quite strange.
And break\_on\_match =\> true is also not working.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [February 20, 2018, 10:36am UTC](https://discuss.elastic.co/t/logstash-grok-duplicate-entries-after-parsing/120563/4 "2018-02-20T10:36:26Z")
Is the following basic example not working?
```auto
grok {
match => {"message" => "%{COMBINEDAPACHELOG} %{QS:trueclientip} %{QS:filetype}"}
}
```
Can you configure a `stdout` output plugin with a `rubydebug` codec and show the result for an event?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 20, 2018, 10:36am UTC](https://discuss.elastic.co/t/logstash-grok-duplicate-entries-after-parsing/120563/5 "2018-03-20T10:36:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.