# Logstash '\_grokparsefailure' issue
**URL:** https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756
**Category:** Logstash
**Created:** [May 11, 2016, 11:29am UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756 "2016-05-11T11:29:26Z")
**Posts on this page:** 15
**Page:** 1
### Author:  [@shubham\_deodia](https://discuss.elastic.co/u/shubham_deodia)
#### Post date: [May 11, 2016, 11:29am UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/1 "2016-05-11T11:29:26Z")
I am using custom grok pattern to parse my log file, Not matter what i do it always give me the \_grokparsefailure exception.
However, it work perfectly on [https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/)
My custom pattern file is located at,
C:\Users\Username\projects\Logstash\bin\patterns
Filename: mylogpattern
> ```
> LogLevel [I|V|D|M|W|E|A|F]
> MODULE \b\w+\b|------
> MESSAGEID (?:[+-]?(?:[0-9]+))|----
> SUBMODULE (.*?:)
> MESSAGE (.*)|(.*?:)|(.*\s*?:)
>
> ```
My Logstash Config File Looks like this:
> ```
> input{
> beats{
> host => "192.168.56.1"
> port => 7088
> congestion_threshold => 200
> }
> }
> filter {
> if [type] == "MyLog"{
> grok{
> patterns_dir => ["C:\Users\Username\projects\Logstash\logstash\bin\patterns"]
> match => { "message" => "%{YEAR:Year}%{MONTHNUM:Month}%{MONTHDAY:Day} %{HOUR:Hour}%{MINUTE:Minute}%{SECOND:Second} %{LogLevel:LogVerbosity} %{MODULE:MODULENAME}%{SPACE}%{MESSAGEID:MESSAGEID} %{SUBMODULE:SUBMODULE} %{MESSAGE:MESSAGE}" }
> add_field => ["received_at", "%{@timestamp}"]
> add_field => ["received_from", "%{host}"]
> }
> }
> }
> output {
> elasticsearch {
> hosts => ["http://localhost:9200"]
> manage_template => false
> index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
> document_type => "%{[@metadata][type]}"
> }
> }
>
> ```
Sample Log File:
> ```
> 20160509 115108 I GEN 0000 ConnectionHandler.reconnect: Reconnect to the DB was done.
>
> ```
> ```
> 20160509 115108 I GEN 0000 84.1.3.1000012 : Reconnect to the DB was done.
>
> ```
It works perfectly on grok debugger, but somehow it is failing while parsing.
tags:beats\_input\_codec\_plain\_applied, \_grokparsefailure
Could someone please help me out with this, What am I doing wrong?
---
### Author:  [@Clement\_Ros](https://discuss.elastic.co/u/Clement_Ros)
#### Post date: [May 11, 2016, 11:37am UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/2 "2016-05-11T11:37:58Z")
Are you sure that log pass through your "if" ?
you can test that by adding "add\_tag =\> "" in your grok.
---
### Author:  [@MrLee](https://discuss.elastic.co/u/MrLee)
#### Post date: [May 11, 2016, 12:10pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/3 "2016-05-11T12:10:46Z")
How do you start your logstash?
Do you have some other config files in the same config directory?
---
### Author:  [@shubham\_deodia](https://discuss.elastic.co/u/shubham_deodia)
#### Post date: [May 11, 2016, 12:23pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/5 "2016-05-11T12:23:49Z")
Starting it using this command, config file is in different folder
> logstash agent -f C:\Users\shubhamd\projects\Logstash\logstash\conf.d\central.conf -l C:\Users\shubhamd\projects\Logstash\logstash\conf.d\lslog.log –-verbose
---
### Author:  [@shubham\_deodia](https://discuss.elastic.co/u/shubham_deodia)
#### Post date: [May 11, 2016, 12:38pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/6 "2016-05-11T12:38:11Z")
When i added the add\_tag statement.
Kibana was showing the correct log for a few seconds
then again, it was displaying somwthing as
> `host:localhost.localdomain tags:beats_input_codec_plain_applied, _grokparsefailure _id:AVSfyfAcoQty0p9BNk2p _type:MyLog _index:filebeat-2016.05.11 _score: -`
Filebeat debug has something like this,
> ```
> 2016/05/11 12:23:32.087717 output.go:87: DBG output worker: publish 1808 events
> 2016/05/11 12:23:32.087761 client.go:146: DBG Try to publish 1808 events to logstash with window size 1024
>
> ```
is it due to the heavy log count.
I have already increased congestion\_threshold =\> 200
What can i do to fix it.
---
### Author:  [@MrLee](https://discuss.elastic.co/u/MrLee)
#### Post date: [May 11, 2016, 12:39pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/7 "2016-05-11T12:39:06Z")
```auto
"It works perfectly on grok debugger, but somehow it is failing while parsing."
```
If so, it must be some log item match failed. If any, it will raise "\_grokparsefailure" error.
Maybe you should check your log data overall or try to deal abnormal situation in logstash.
---
### Author:  [@shubham\_deodia](https://discuss.elastic.co/u/shubham_deodia)
#### Post date: [May 11, 2016, 12:43pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/8 "2016-05-11T12:43:12Z")
All log follows essentially the same format, also i have added GREEDYDATA filter for the long messages, it is able to parse it successfully. I can still see some parsed logs in kibana.
Its just the log count is really high and log file also have continuous blank lines in between.
Does it try to parse those blank lines as well ?
---
### Author:  [@Clement\_Ros](https://discuss.elastic.co/u/Clement_Ros)
#### Post date: [May 11, 2016, 12:43pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/9 "2016-05-11T12:43:42Z")
> [@shubham\_deodia](#):
>
> host:localhost.localdomain tags:beats\_input\_codec\_plain\_applied, \_grokparsefailure \_id:AVSfyfAcoQty0p9BNk2p \_type:MyLog \_index:filebeat-2016.05.11 \_score: -
We don't see the new tag, so the problem must be your condition `if [type] == "MyLog"`.
can you try to replace `if [type] == "MyLog"` by `if "MyLog" in [type]` ?
---
### Author:  [@shubham\_deodia](https://discuss.elastic.co/u/shubham_deodia)
#### Post date: [May 11, 2016, 12:47pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/10 "2016-05-11T12:47:45Z")
> tags:beats\_input\_codec\_plain\_applied, **Groked** message:20160502 092820 I BL 0003 284.1.3.1000051 : : No new files waiting in AC for collection @version:1 @timestamp:May 11th 2016, 17:52:23.705 type:Gatherer beat.hostname:localhost.localdomain beat.name:localhost.localdomain source:/var/log/log/mylog.log offset:583,680 input\_type:log count:1 fields: - host:localhost.localdomain Year:2016 Month:05 Day:02 Hour:09 Minute:28 Second:20 LogVerbosity:I MODULENAME:BL MESSAGEID:0003 SUBMODULE:284.1.3.1000051 : MESSAGE:ROAMING\_VOICE\_AC\_COLLECTOR\_1000051: logmessage\_at:2016-05-11T12:22:23.705Z
this is the successfully parsed log output in kibana, i can see it has Groked tag.
But still , logs which came after fe seconds are still not getting parse.
What to do ?
---
### Author:  [@MrLee](https://discuss.elastic.co/u/MrLee)
#### Post date: [May 11, 2016, 12:57pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/11 "2016-05-11T12:57:34Z")
@Clement_Ros
In my opinion,If the "if" condition is not satisfied, it will not pass though the grok filter, will not raise a "\_grokparsefailure" error.
---
### Author:  [@Clement\_Ros](https://discuss.elastic.co/u/Clement_Ros)
#### Post date: [May 11, 2016, 12:59pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/12 "2016-05-11T12:59:03Z")
you are right @MrLee ...
---
### Author:  [@Eric\_Amick](https://discuss.elastic.co/u/Eric_Amick)
#### Post date: [May 11, 2016, 1:03pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/13 "2016-05-11T13:03:43Z")
It won't solve your problem, but that LogLevel pattern matches the "|" character, which I'm sure you didn't intend. It should be [IVDMWEAF] instead.
---
### Author:  [@shubham\_deodia](https://discuss.elastic.co/u/shubham_deodia)
#### Post date: [May 11, 2016, 1:05pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/14 "2016-05-11T13:05:58Z")
Thanks Eric, Made the correction.
Still no luck with parsefailure error 😖
---
### Author:  [@shubham\_deodia](https://discuss.elastic.co/u/shubham_deodia)
#### Post date: [May 11, 2016, 1:22pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/15 "2016-05-11T13:22:56Z")
So , I limited my log file to log count of around 800 and it worked.
Will be increasing more counts and do the load testing.
I am using filebeat as log shipper.
Just in case if it is the issue, does anyone know what changes needs to be made in any BELK , in order at to handle high frequency of logs.
Appreciate any help you could provide.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:58am UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-issue/49756/16 "2017-07-06T04:58:12Z")