# Grok works on Grok debugger but got Grokparsefailure in logstash

**URL:** <https://discuss.elastic.co/t/grok-works-on-grok-debugger-but-got-grokparsefailure-in-logstash/281227>\
**Category:** Logstash\
**Created:** [August 12, 2021, 3:45pm UTC](https://discuss.elastic.co/t/grok-works-on-grok-debugger-but-got-grokparsefailure-in-logstash/281227 "2021-08-12T15:45:53Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![syunusic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syunusic/32/4131_2.png) [@syunusic](https://discuss.elastic.co/u/syunusic)\
**Post date:** [August 12, 2021, 3:45pm UTC](https://discuss.elastic.co/t/grok-works-on-grok-debugger-but-got-grokparsefailure-in-logstash/281227/1 "2021-08-12T15:45:53Z")

</div>

If I go to [http://grokdebug.herokuapp.com/](http://grokdebug.herokuapp.com/) or the GROK debugger in Kibana and use this sample and this grok, works fine.. but if I run it in logstash, I get a grokfailure tag:  
Sample:

```auto
NSX 2281528 - [nsx@6876 comp=\"nsx-esx\" subcomp=\"mpa-client\" tid=\"2281586\" level=\"INFO\"] [AggSvc-L2-Bridging] SendRequest: To Master APH, Publish, type (com.vmware.nsx.management.aggservice.l2.HostLogicalPortStatusMsg) correlationId () Success.

```

and this is the grok:

```auto
NSX (?<nsx_numero>[^]+) - \[nsx@(?<nsx_numero_despuesdelaarroba>[^]+) comp=\\\"(?<nsx_comp>[^\\]+)\\\" subcomp=\\\"(?<nsx_subcomp>[^\\]+)\\\" tid=\\\"(?<nsx_tid>[^\\]+)\\\" level=\\\"(?<nsx_level>[^\\]+)\\\"\] \[(?<nsx_blah1>[^\]]+)\](\[(?<nsx_blah2>[^\]]+)\]|)(|:) %{GREEDYDATA:nsx_mensaje_final}

```

Where is the problem?  
I’m positive is related to the escapes, but I can't find the problem.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 12, 2021, 3:57pm UTC](https://discuss.elastic.co/t/grok-works-on-grok-debugger-but-got-grokparsefailure-in-logstash/281227/2 "2021-08-12T15:57:10Z")

</div>

```
input { generator { count => 1 lines => ['NSX 2281528 - [nsx@6876 comp=\"nsx-esx\" subcomp=\"mpa-client\" tid=\"2281586\" level=\"INFO\"] [AggSvc-L2-Bridging] SendRequest: To Master APH, Publish, type (com.vmware.nsx.management.aggservice.l2.HostLogicalPortStatusMsg) correlationId () Success.' ] } }
filter {
    grok { match => { "message" => "NSX (?<nsx_numero>[^]+) - \[nsx@(?<nsx_numero_despuesdelaarroba>[^]+) comp=\\\"(?<nsx_comp>[^\\]+)\\\" subcomp=\\\"(?<nsx_subcomp>[^\\]+)\\\" tid=\\\"(?<nsx_tid>[^\\]+)\\\" level=\\\"(?<nsx_level>[^\\]+)\\\"\] \[(?<nsx_blah1>[^\]]+)\](\[(?<nsx_blah2>[^\]]+)\]|)(|:) %{GREEDYDATA:nsx_mensaje_final}" } }
}
output { stdout { codec => rubydebug { metadata => false } } }

```

works just fine for me

```
                   "nsx_level" => "INFO",
                   "nsx_blah1" => "AggSvc-L2-Bridging",
                    "nsx_comp" => "nsx-esx",
                 "nsx_subcomp" => "mpa-client",
                  "nsx_numero" => "2281528",

```

etc.

---

<div class="post-metadata">

**Author:** ![syunusic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syunusic/32/4131_2.png) [@syunusic](https://discuss.elastic.co/u/syunusic)\
**Post date:** [August 12, 2021, 5:57pm UTC](https://discuss.elastic.co/t/grok-works-on-grok-debugger-but-got-grokparsefailure-in-logstash/281227/3 "2021-08-12T17:57:41Z")

</div>

If I run exactly your configuration works fine.. so the problem is not the grok itself. This is my configuration where the problem occurs:

```auto
input
{
file { path => "/tmp/lala.json" start_position => "beginning" ignore_older => 15552000 codec => "json" }

}
filter
{
        grok { match => { "syslog_message" => "NSX (?<nsx_numero>[^]+) - \[nsx@(?<nsx_numero_despuesdelaarroba>[^]+) comp=\\\"(?<nsx_comp>[^\\]+)\\\" subcomp=\\\"(?<nsx_subcomp>[^\\]+)\\\" tid=\\\"(?<nsx_tid>[^\\]+)\\\" level=\\\"(?<nsx_level>[^\\]+)\\\"\] \[(?<nsx_blah1>[^\]]+)\](\[(?<nsx_blah2>[^\]]+)\]|)(|:) %{GREEDYDATA:nsx_mensaje_final}" } }
}
output
{
stdout { codec => rubydebug }
}

```

And this is the lala.json file:

```auto
{"product":"vmware","@version":"1","@timestamp":"2021-08-12T06:20:50.000Z","client":"client","hostname":"esxi","program":"nsx-exporter","message":"Aug 12 02:20:50 esxi nsx-exporter: NSX 2281528 - [nsx@6876 comp=\"nsx-esx\" subcomp=\"mpa-client\" tid=\"2281586\" level=\"INFO\"] [AggSvc-L2-Bridging] SendRequest: To Master APH, Publish, type (com.vmware.nsx.management.aggservice.l2.BridgeEndpointsOnBridgeNodeMsg) correlationId () Success.","type":"syslog","syslog_message":"NSX 2281528 - [nsx@6876 comp=\"nsx-esx\" subcomp=\"mpa-client\" tid=\"2281586\" level=\"INFO\"] [AggSvc-L2-Bridging] SendRequest: To Master APH, Publish, type (com.vmware.nsx.management.aggservice.l2.BridgeEndpointsOnBridgeNodeMsg) correlationId () Success.","message_program":"nsx-exporter","host":"data"}

```

Any ideas on where can be the problem?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 12, 2021, 6:17pm UTC](https://discuss.elastic.co/t/grok-works-on-grok-debugger-but-got-grokparsefailure-in-logstash/281227/4 "2021-08-12T18:17:00Z")

</div>

> "syslog\_message":"NSX 2281528 - [nsx@6876 comp="nsx-esx" subcomp="mpa-client" tid="2281586" level="INFO"]

The double quotes within syslog\_message are escaped so that they do not terminate the value of the field. They are not really there. So

```
tid=\\\"(?<nsx_tid>[^\\]+)\\\"

```

should be

```
tid="(?<nsx_tid>[^"]+)"

```

etc.

---

<div class="post-metadata">

**Author:** ![syunusic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syunusic/32/4131_2.png) [@syunusic](https://discuss.elastic.co/u/syunusic)\
**Post date:** [August 12, 2021, 6:47pm UTC](https://discuss.elastic.co/t/grok-works-on-grok-debugger-but-got-grokparsefailure-in-logstash/281227/5 "2021-08-12T18:47:00Z")

</div>

Your answer was on the right track, but not quite what you said. At the end this is what I nedded to do: Not to "double" escape everything, but was neccesary to escape it once... if that makes any sense.. this is the grok that worked at the end:

```auto
grok { match => { "syslog_message" => "NSX (?<nsx_numero>[^]+) - \[nsx@(?<nsx_numero_despuesdelaarroba>[^]+) comp=\"(?<nsx_comp>[^\"]+)\" subcomp=\"(?<nsx_subcomp>[^\"]+)\" tid=\"(?<nsx_tid>[^\"]+)\" level=\"(?<nsx_level>[^\"]+)\"\] \[(?<nsx_blah1>[^\]]+)\](\[(?<nsx_blah2>[^\]]+)\]|)(|:) %{GREEDYDATA:nsx_mensaje_final}" } }

```

so, in your example would be:

```auto
tid=\"(?<nsx_tid>[^\"]+)\"

```

Thanks for your help!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 12, 2021, 6:55pm UTC](https://discuss.elastic.co/t/grok-works-on-grok-debugger-but-got-grokparsefailure-in-logstash/281227/6 "2021-08-12T18:55:34Z")

</div>

> [@syunusic](#):
>
> Not to "double" escape everything, but was neccesary to escape it once

Oh, yeah, of course. 😳 Otherwise the " terminates the grok pattern and logstash would complain.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 9, 2021, 6:55pm UTC](https://discuss.elastic.co/t/grok-works-on-grok-debugger-but-got-grokparsefailure-in-logstash/281227/7 "2021-09-09T18:55:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.