# Logstash - Grok parse failure

**URL:** <https://discuss.elastic.co/t/logstash-grok-parse-failure/138403>\
**Category:** Logstash\
**Created:** [July 3, 2018, 1:44pm UTC](https://discuss.elastic.co/t/logstash-grok-parse-failure/138403 "2018-07-03T13:44:26Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Aby](https://avatars.discourse-cdn.com/v4/letter/a/a88e57/32.png) [@Aby](https://discuss.elastic.co/u/Aby)\
**Post date:** [July 3, 2018, 1:44pm UTC](https://discuss.elastic.co/t/logstash-grok-parse-failure/138403/1 "2018-07-03T13:44:26Z")

</div>

Hi - i am trying to parse the below log file:

[2018-05-29 08:04:23,687] DEBUG - http-outgoing-2845 \<\< HTTP/1.1 500 Internal Server Error {org.apache.synapse.transport.http.headers}  
[2018-05-29 08:04:18,037] DEBUG - http-outgoing-2842 \<\< HTTP/1.1 200 OK {org.apache.synapse.transport.http.headers}

below is my conf

input {  
file {  
path =\> "/home/abhi/ipclick.log"  
start\_position=\> "beginning"  
}  
}

filter {  
grok {  
break\_on\_match =\> false  
match =\>  
{'message' =\> ['%{SYSLOG5424SD:date} DEBUG - http-outgoing%{BASE16FLOAT:outputid} \<\< HTTP%{URIPATHPARAM:URL} 500 %{GREEDYDATA:STATUS} {%{JAVACLASS:MSG}}','%{SYSLOG5424SD:date} DEBUG - \>\> %{QS:STATUS} {%{JAVACLASS:MSG}}','%{SYSLOG5424SD:date} DEBUG - http-outgoing%{BASE16FLOAT:outputid} \>\> %{GREEDYDATA: request} %{URIPATHPARAM:URL} {%{JAVACLASS:MSG}}'  
,'%{SYSLOG5424SD:date} DEBUG - http-outgoing%{BASE16FLOAT:outputid} \<\< HTTP%{URIPATHPARAM:URL} 200 %{GREEDYDATA:STATUS} {%{JAVACLASS:MSG}}']}  
}

#if "\_grokparsefailure" in [tags] {

# drop{}

#}  
}  
output {  
elasticsearch {  
hosts =\> "XXXXX:9200"  
index =\> "log"  
}  
stdout {  
codec =\> rubydebug  
}  
}

I get the below output: [even though my message pattern is same for output 1 & 2, my second output parses the value but with a \_grokparsefailure issue.

grok filter pattern that i have used in both the cases are same. Any help will be appreciated..

{  
"URL" =\> "/1.1",  
"path" =\> "/home/abhi/ipclick.log",  
"MSG" =\> "org.apache.synapse.transport.http.headers",  
"host" =\> "SCPUBU",  
"date" =\> "[2018-05-29 08:04:18,037]",  
"outputid" =\> "-2842",  
**_"message" =\> "[2018-05-29 08:04:18,037] DEBUG - http-outgoing-2842 \<\< HTTP/1.1 200 OK {org.apache.synapse.transport.http.headers}",_**  
"STATUS" =\> "OK",  
"@timestamp" =\> 2018-07-03T09:59:22.087Z,  
"@version" =\> "1"  
}  
{  
"URL" =\> "/1.1",  
"outputid" =\> "-2845",  
"date" =\> "[2018-06-29 08:04:23,687]",  
_**"message" =\> "[2018-06-29 08:04:23,687] DEBUG - http-outgoing-2845 \<\< HTTP/1.1 500 OK {org.apache.synapse.tranr}"**_,  
"@timestamp" =\> 2018-07-03T09:59:22.086Z,  
"@version" =\> "1",  
"MSG" =\> "org.apache.synapse.tranr",  
"path" =\> "/home/abhi/ipclick.log",  
"host" =\> "SCPUBU",  
"STATUS" =\> "OK",  
"tags" =\> [  
[0] "\_grokparsefailure"  
]  
}

---

<div class="post-metadata">

**Author:** ![Aby](https://avatars.discourse-cdn.com/v4/letter/a/a88e57/32.png) [@Aby](https://discuss.elastic.co/u/Aby)\
**Post date:** [July 3, 2018, 4:33pm UTC](https://discuss.elastic.co/t/logstash-grok-parse-failure/138403/2 "2018-07-03T16:33:05Z")

</div>

Just to add:

In our case, there are 2 patterns defined in the array, only the second pattern works. 1st pattern throws a grok parse failure though it parses correctly without issue and this is the case if it's vice versa.

1st pattern always comes with the \_grokparsefailure error

'message' =\> ['%{SYSLOG5424SD:date} DEBUG - http-outgoing-%{BASE16FLOAT:outputid} \<\< HTTP%{URIPATHPARAM:URL} %{GREEDYDATA:STATUS}{%{JAVACLASS:MSG}}','%{SYSLOG5424SD:date} DEBUG - \>\> %{QS:STATUS} {%{JAVACLASS:MSG}}']

Output:  
{  
"@version" =\> "1",  
"host" =\> "SCPUBU",  
"STATUS" =\> "200 OK ",  
"URL" =\> "/1.1",  
"MSG" =\> "org.apache.synapse.transport.http.headers",  
"tags" =\> [  
[0] "\_grokparsefailure"  
],  
"@timestamp" =\> 2018-07-03T16:27:32.822Z,  
"date" =\> "[2018-05-29 08:04:18,037]",  
"path" =\> "/home/abhi/test.log",  
"message" =\> "[2018-05-29 08:04:18,037] DEBUG - http-outgoing-2842 \<\< HTTP/1.1 200 OK {org.apache.synapse.transport.http.headers}",  
"outputid" =\> "2842"  
}

{  
"@version" =\> "1",  
"date" =\> "[2018-05-29 08:04:11,797]",  
"host" =\> "SCPUBU",  
"STATUS" =\> ""OPTIONS /intrCaco2.1.1/2.1.1/api/service/predict HTTP/1.1[\r][\n]"",  
"path" =\> "/home/abhi/test.log",  
"message" =\> "[2018-05-29 08:04:11,797] DEBUG - \>\> "OPTIONS /intrCaco2.1.1/2.1.1/api/service/predict HTTP/1.1[\r][\n]" {org.apache.synapse.transport.http.wire}",  
"MSG" =\> "org.apache.synapse.transport.http.wire",  
"@timestamp" =\> 2018-07-03T16:27:32.822Z  
}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 31, 2018, 4:33pm UTC](https://discuss.elastic.co/t/logstash-grok-parse-failure/138403/3 "2018-07-31T16:33:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.