# Logstash \_grokparsefailure . Unable to find issue **URL:** \ **Category:** Logstash\ **Created:** [September 29, 2016, 6:58pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845 "2016-09-29T18:58:27Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![chromechris](https://avatars.discourse-cdn.com/v4/letter/c/e47c2d/32.png) [@chromechris](https://discuss.elastic.co/u/chromechris)\ **Post date:** [September 29, 2016, 6:58pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/1 "2016-09-29T18:58:27Z")
Hello Loggers, I have been receiveing a \_grokparsefailure tag in Logstash but cannot find the issue. My Logstash Grok filter looks like this: grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp}%{GREEDYDATA:input}" } } I have no idea what the issue is. My log contains an ISO8601 timestamp "2016-09-28T09:44:22,533-0700 data-blah blah \r" and a carriage return character at the end that is not visible. Should'nt GREEDYDATA pickup the carriage return symbol as well? I have also tried using my timestamp with a space intead of T "2016-09-28 09:44:22,533-0700" but I get the same issue. The space between the timestamp and GREDDYDATA exist, which is why a have a space after my timestamp in my grok filter. Thanks in advance guys, chromehris. ---
**Author:** ![chromechris](https://avatars.discourse-cdn.com/v4/letter/c/e47c2d/32.png) [@chromechris](https://discuss.elastic.co/u/chromechris)\ **Post date:** [September 29, 2016, 7:01pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/2 "2016-09-29T19:01:28Z")
Also, I want to mention that I also have a KV filter and a Mutate filter, but I believe "\_grokparsefailure" is only caused by a grok filter correct? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 29, 2016, 8:30pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/3 "2016-09-29T20:30:15Z")
Works for me: ```plaintext $ cat test.config input { stdin { } } output { stdout { codec => rubydebug } } filter { grok { match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}%{GREEDYDATA:input}" } } } $ echo "2016-09-28T09:44:22,533-0700 data-blah blah\r" | logstash -f test.config Settings: Default pipeline workers: 8 Pipeline main started { "message" => "2016-09-28T09:44:22,533-0700 data-blah blah\r", "@version" => "1", "@timestamp" => "2016-09-29T20:29:50.414Z", "host" => "bertie", "timestamp" => "2016-09-28T09:44:22,533-0700", "input" => " data-blah blah\r" } Pipeline main has been shutdown stopping pipeline {:id=>"main"} ``` > but I believe "\_grokparsefailure" is only caused by a grok filter correct? Yes. ---
**Author:** ![chromechris](https://avatars.discourse-cdn.com/v4/letter/c/e47c2d/32.png) [@chromechris](https://discuss.elastic.co/u/chromechris)\ **Post date:** [September 29, 2016, 8:37pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/4 "2016-09-29T20:37:45Z")
Thanks Magnus! Man, sorry about that, I forgot to place a space between TImestamp & Greedydata. grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp}%{GREEDYDATA:input}" } } Also, the "\r" is invisible as text. It's a "Carriage Return" special character. I am thinking it is this special character that gives the issue. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 30, 2016, 5:31am UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/5 "2016-09-30T05:31:01Z")
> Also, the "\r" is invisible as text. It's a "Carriage Return" special character. Yes, of course. > I am thinking it is this special character that gives the issue. What issue? You may want to use the mutate filter's strip option to remove the trailing carriage return. ---
**Author:** ![chromechris](https://avatars.discourse-cdn.com/v4/letter/c/e47c2d/32.png) [@chromechris](https://discuss.elastic.co/u/chromechris)\ **Post date:** [September 30, 2016, 4:10pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/6 "2016-09-30T16:10:45Z")
This is my full filter which removes the "\r" carraige return special character: filter { if [type] == "businesslog" { grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} %{GREEDYDATA:input}" } } kv { source =\> "input" field\_split =\> "|" value\_split =\> "=" } mutate { remove\_field =\> ["input"] gsub =\> [ "\*", "\r", "" ] } } } Then, I tried removing the special character before my grok filter: filter { if [type] == "businesslog" { if [message] =~ "\r" { mutate { gsub =\> ["_", "\r", ""] } } grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} %{GREEDYDATA:input}" } } kv { source =\> "input" field\_split =\> "|" value\_split =\> "=" } # mutate { # remove\_field =\> ["input"] # gsub =\> [ # "_", "\r", "" # ] # } } } I still get a grok parse failure. My logs look like: 2016-09-28 12:44:22,533-0700 key=00000|key=9392000|key=99383|key=7438932883\r The "\r" at the end symbolizes the special character. I am not sure what could be cuasing this \_grokparsefailure anymore. ---
**Author:** ![chromechris](https://avatars.discourse-cdn.com/v4/letter/c/e47c2d/32.png) [@chromechris](https://discuss.elastic.co/u/chromechris)\ **Post date:** [September 30, 2016, 4:16pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/7 "2016-09-30T16:16:46Z")
Also tried this filter, in case the message field did not exists yet: filter { if [type] == "businesslog" { # if [message] =~ "\r" { mutate { gsub =\> ["_", "\r", ""] } # } grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} %{GREEDYDATA:input}" } } kv { source =\> "input" field\_split =\> "|" value\_split =\> "=" } # mutate { # remove\_field =\> ["input"] # gsub =\> [ # "_", "\r", "" # ] # } } } ---
**Author:** ![chromechris](https://avatars.discourse-cdn.com/v4/letter/c/e47c2d/32.png) [@chromechris](https://discuss.elastic.co/u/chromechris)\ **Post date:** [September 30, 2016, 4:44pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/8 "2016-09-30T16:44:10Z")
And this one, specifically stating the message field in gsub before the grok filter: filter { if [type] == "businesslog" { #if [message] =~ "\r" { mutate { gsub =\> ["message", "\r", ""] } # } grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} %{GREEDYDATA:input}" } } kv { source =\> "input" field\_split =\> "|" value\_split =\> "=" } # mutate { # remove\_field =\> ["input"] # gsub =\> [ # "\*", "\r", "" # ] # } } } After all these attempts, I still receive a \_grokparsefailure ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 30, 2016, 5:34pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/9 "2016-09-30T17:34:33Z")
Did you try the mutate filter's strip option? ---
**Author:** ![chromechris](https://avatars.discourse-cdn.com/v4/letter/c/e47c2d/32.png) [@chromechris](https://discuss.elastic.co/u/chromechris)\ **Post date:** [September 30, 2016, 5:40pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/10 "2016-09-30T17:40:06Z")
Isn't strip only for whitespace & newline? I tried: filter { if [type] == "businesslog" { #if [message] =~ "\r" { mutate { strip =\> ["message"] } # mutate { # gsub =\> ["message", "\r", ""] # } # } grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} %{GREEDYDATA:input}" } } kv { source =\> "input" field\_split =\> "|" value\_split =\> "=" } mutate { remove\_field =\> ["input"] #gsub =\> [ # "\*", "\r", "" #] } } } Still \_grokparsefailure Could it be because the "message" field does not exists yet at the time that I am using the mutate filter on it? ---
**Author:** ![chromechris](https://avatars.discourse-cdn.com/v4/letter/c/e47c2d/32.png) [@chromechris](https://discuss.elastic.co/u/chromechris)\ **Post date:** [September 30, 2016, 6:54pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/11 "2016-09-30T18:54:25Z")
I have even removed the "/r" from my logs, so I changed my filter to: filter { if [type] == "businesslog" { mutate { strip =\> ["message"] #gsub =\> ["message", "\r", ""] } grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} %{GREEDYDATA:input}" } } kv { source =\> "input" field\_split =\> "|" value\_split =\> "=" } mutate { remove\_field =\> ["input"] #gsub =\> [ # "\*", "\r", "" #] } } } I still get a \_grokparsefailure, even though all my data is coming in accurately. It'sjust uncomftrable to have that \_grokparsefailure. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 1, 2016, 3:41pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/12 "2016-10-01T15:41:49Z")
Are you sure you don't have any other grok filters laying around in your configuration files? Remember that Logstash reads _all_ files in /etc/logstash/conf.d. As I showed in a previous example the grok filter _does_ work and does _not_ add a `_grokparsefailure` tag. It seems you're either not showing us all of your configuration or you're not using the configuration you think you are. Replicating my example could be a good start. ---
**Author:** ![chromechris](https://avatars.discourse-cdn.com/v4/letter/c/e47c2d/32.png) [@chromechris](https://discuss.elastic.co/u/chromechris)\ **Post date:** [October 5, 2016, 9:18pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/13 "2016-10-05T21:18:09Z")
I indeed am using multiple config files under /conf.d, one is an output and the others are input-&-filter each . My configuration however passes the configtest. How can multiple configuration files affect a \_grokparsefailure tag in a certain message that comes from a config file which does not create such a tag? Since I most certainly know that my configuration file that I have originally mentioned should not cause a \_grokparsefailure. I have tries different filters and I still receive this \_grokparsefailure, so this tag must be coming from another source. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 6, 2016, 5:39am UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/14 "2016-10-06T05:39:09Z")
> My configuration however passes the configtest. That basically just means it's syntactically correct. Whether it does what you expect it to is a very different matter. > How can multiple configuration files affect a \_grokparsefailure tag in a certain message that comes from a config file which does not create such a tag? Another file could contain a grok filter that isn't under a condition, meaning it applies to all events. file A: ```plaintext filter { grok { match => ["message", "this pattern doesn't match anything"] } } ``` file B: ```plaintext filter { if [type] == "sometype" { grok { match => ["message", "this pattern matches my log file"] } } } ``` In this example, every single event will get the `_grokparsefailure` tag since the grok filter in file A will apply to all events but will match none of them. ---
**Author:** ![chromechris](https://avatars.discourse-cdn.com/v4/letter/c/e47c2d/32.png) [@chromechris](https://discuss.elastic.co/u/chromechris)\ **Post date:** [October 6, 2016, 9:02pm UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/15 "2016-10-06T21:02:34Z")
> [@magnusbaeck](#): > > he grok filter in file If I have: File A filter { if [app] == "app1" { mutate { add\_field =\> { "type" =\> "type1" } } grok { match =\> ["message", "this pattern matches my log file"] } } } File B filter { if [app] == "app2" { mutate { add\_field =\> { "type" =\> "type1" } } grok { match =\> ["message", "this pattern matches my log file"] } } } Will there be any issues with both of these having the same type, even though I first split them by a different "app" field value? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 7, 2016, 5:46am UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/16 "2016-10-07T05:46:51Z")
This should be okay, but keep in mind that `add_field` appends to any existing values so you'll probably end up with a `type` field that's a two-element array. Prefer the mutate filter's `replace` option. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:35am UTC](https://discuss.elastic.co/t/logstash--grokparsefailure-unable-to-find-issue/61845/17 "2017-07-06T04:35:18Z")