# Grok debugger in dev tools parses correctly, fails running logstash

**URL:** <https://discuss.elastic.co/t/grok-debugger-in-dev-tools-parses-correctly-fails-running-logstash/199087>\
**Category:** Logstash\
**Created:** [September 11, 2019, 2:12pm UTC](https://discuss.elastic.co/t/grok-debugger-in-dev-tools-parses-correctly-fails-running-logstash/199087 "2019-09-11T14:12:30Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![starrynight](https://avatars.discourse-cdn.com/v4/letter/s/a5b964/32.png) [@starrynight](https://discuss.elastic.co/u/starrynight)\
**Post date:** [September 11, 2019, 2:12pm UTC](https://discuss.elastic.co/t/grok-debugger-in-dev-tools-parses-correctly-fails-running-logstash/199087/1 "2019-09-11T14:12:30Z")

</div>

I was attempting to parse an atlassian bitbucket log here -- with the log example being this:

`10.10.10.2,10.11.1.2 | https | i@5EB8GFx642x581467x0 | - | 2019-08-27 10:42:28,325 | "GET /security_logout HTTP/1.1" | "" "Zabbix" | - | - | - | - | - | 18oho66 |`

...which ended up coming up with the following grok statement using the dev tools grok debugger in kibana (on ver 7.3.0) and showed as correct:

`%{IPV4:client_ip},%{IPV4:proxy_ip} \| %{DATA:protocol} \| %{DATA:request_id} \| %{HTTPDUSER:user} \| %{TIMESTAMP_ISO8601:datelog} \| "%{DATA:action} %{DATA:resource} %{DATA:http_version}" \| "" "%{DATA:request_details}" \| %{DATA:response} \| %{DATA:bytes_read} \| %{DATA:bytes_written} \| %{DATA:labels} \| %{DATA:response_time} \| %{DATA:session_id} \|`

Upon adding it in the correct format in logstash conf.d dir and restarting it fails on compile and on checking it on the grok debugger at [herokuapp.com](http://herokuapp.com), it is showing with a compile error as well. Confused...

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 11, 2019, 2:47pm UTC](https://discuss.elastic.co/t/grok-debugger-in-dev-tools-parses-correctly-fails-running-logstash/199087/2 "2019-09-11T14:47:07Z")

</div>

Please edit your post, select the log message and click on \</\> in the toolbar above the edit pane. Make sure that the appearence of the log message changes to be

```
like this

```

Then do the same for the grok pattern.

---

<div class="post-metadata">

**Author:** ![starrynight](https://avatars.discourse-cdn.com/v4/letter/s/a5b964/32.png) [@starrynight](https://discuss.elastic.co/u/starrynight)\
**Post date:** [September 16, 2019, 8:19am UTC](https://discuss.elastic.co/t/grok-debugger-in-dev-tools-parses-correctly-fails-running-logstash/199087/3 "2019-09-16T08:19:27Z")

</div>

I've done this, thanks 🙂

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 16, 2019, 2:59pm UTC](https://discuss.elastic.co/t/grok-debugger-in-dev-tools-parses-correctly-fails-running-logstash/199087/4 "2019-09-16T14:59:46Z")

</div>

What error message does logstash output? That correctly parses the input line for me.

---

<div class="post-metadata">

**Author:** ![starrynight](https://avatars.discourse-cdn.com/v4/letter/s/a5b964/32.png) [@starrynight](https://discuss.elastic.co/u/starrynight)\
**Post date:** [September 18, 2019, 12:01pm UTC](https://discuss.elastic.co/t/grok-debugger-in-dev-tools-parses-correctly-fails-running-logstash/199087/5 "2019-09-18T12:01:56Z")

</div>

This is what I'm getting:

`Sep 18 13:57:20 hostname logstash[22451]: [2019-09-18T13:57:20,972][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, {, } at line 106, column 166 (byte 2739) after filter {\n if [id] == [\"atlbbaccess\"] {\n grok {\n match => {\"message\" => \"^%{IPV4:client_ip},%{IPV4:proxy_ip} \\| %{DATA:protocol} \\| %{DATA:request_id} \\| %{HTTPDUSER:user} \\| %{TIMESTAMP_ISO8601:datelog} \\| \"", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:41:in`compile\_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in`block in compile\_sources'", "org/jruby/RubyArray.java:2577:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in`compile\_sources'", "org/logstash/execution/AbstractPipelineExt.java:151:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in`initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java\_pipeline.rb:24:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in`execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:325:in `block in converge_state'"]}`

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 18, 2019, 12:53pm UTC](https://discuss.elastic.co/t/grok-debugger-in-dev-tools-parses-correctly-fails-running-logstash/199087/6 "2019-09-18T12:53:09Z")

</div>

> [@starrynight](#):
>
> %{TIMESTAMP\_ISO8601:datelog} | "%{DATA:action}

You need to escape the double quotes, or, as I did, use single quotes around the grok pattern.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 16, 2019, 12:53pm UTC](https://discuss.elastic.co/t/grok-debugger-in-dev-tools-parses-correctly-fails-running-logstash/199087/7 "2019-10-16T12:53:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.