# Cannot use example Grok Pattern (official docs) in Grok Processor

**URL:** <https://discuss.elastic.co/t/cannot-use-example-grok-pattern-official-docs-in-grok-processor/358871>\
**Category:** Kibana\
**Created:** [May 6, 2024, 8:53pm UTC](https://discuss.elastic.co/t/cannot-use-example-grok-pattern-official-docs-in-grok-processor/358871 "2024-05-06T20:53:06Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![paolovalladolid](https://avatars.discourse-cdn.com/v4/letter/p/edb3f5/32.png) [@paolovalladolid](https://discuss.elastic.co/u/paolovalladolid)\
**Post date:** [May 6, 2024, 8:53pm UTC](https://discuss.elastic.co/t/cannot-use-example-grok-pattern-official-docs-in-grok-processor/358871/1 "2024-05-06T20:53:06Z")

</div>

This is the pattern I'm trying to use in the Grok processor:

```auto
%{IPORHOST:source.ip} %{USER:user.id} %{USER:user.name} \[%{HTTPDATE:@timestamp}\] "%{WORD:http.request.method} %{DATA:url.original} HTTP/%{NUMBER:http.version}" %{NUMBER:http.response.status_code:int} (?:-|%{NUMBER:http.response.body.bytes:int}) %{QS:http.request.referrer} %{QS:user_agent}

```

The pattern is lifted straight off the official docs here:

> **[Example: Parse logs in the Common Log Format | Elasticsearch Guide \[8.13\] |...](https://www.elastic.co/guide/en/elasticsearch/reference/current/common-log-format-example.html)**

But when I try to add the processor to my Ingest Pipeline, the pattern is highlighted red and I do not get a Save button

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/5/e5b34d3ade44a69066aba6cae559dc1c53338262.png)

Same pattern works fine in Grok Debugger in Dev Tools

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/b/bb6bd9b5f8c735e092c9d93f9f9641a21b7ea7b2.png)

Something about HTTPDATE is breaking the processor in Ingest Pipeline. This cut down pattern still causes an issue

`%{IPORHOST:source.ip} %{USER:user.id} %{USER:user.name} \[%{HTTPDATE:thetime}\]`

It seems like the Grok processor does not recognize `\` as an escape character when running inside Ingest Pipeline. Is there an alternative escape character or pattern?

---

<div class="post-metadata">

**Author:** ![paolovalladolid](https://avatars.discourse-cdn.com/v4/letter/p/edb3f5/32.png) [@paolovalladolid](https://discuss.elastic.co/u/paolovalladolid)\
**Post date:** [May 6, 2024, 9:54pm UTC](https://discuss.elastic.co/t/cannot-use-example-grok-pattern-official-docs-in-grok-processor/358871/2 "2024-05-06T21:54:14Z")

</div>

We are actually on 7.17. Unfortunately I have no control over when, if ever, we will upgrade to Kibana 8.x.

I took the grok pattern from here:

> **[Example: Parse logs in the Common Log Format | Elasticsearch Guide \[7.17\] |...](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/common-log-format-example.html)**

It does not cause JSON format errors like the 8.x pattern. However, the processor keeps getting skipped. This is the condition:

```auto
ctx?.fields?.tomcat_app_id != null && ctx.fields.tomcat_app_id == 'ams-cache-manager-ingress'"

```

This is the relevant portion of the document

`"tomcat_app_id": "ams-cache-manager-ingress"`

This is the verbose output from the pipeline after I run it, stating the processor was skipped.

```auto
        {
          "processor_type": "grok",
          "status": "skipped",
          "if": {
            "condition": "ctx?.fields?.tomcat_app_id != null && ctx.fields.tomcat_app_id == 'ams-cache-manager-ingress'",
            "result": false
          }
        }

```

The document clearly shows tomcat\_app\_id is the correct value. I don't get why the processor would think it does not match.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [May 7, 2024, 1:03am UTC](https://discuss.elastic.co/t/cannot-use-example-grok-pattern-official-docs-in-grok-processor/358871/3 "2024-05-07T01:03:57Z")

</div>

Can you share a sample of your document that is not working?

---

<div class="post-metadata">

**Author:** ![paolovalladolid](https://avatars.discourse-cdn.com/v4/letter/p/edb3f5/32.png) [@paolovalladolid](https://discuss.elastic.co/u/paolovalladolid)\
**Post date:** [May 7, 2024, 2:43pm UTC](https://discuss.elastic.co/t/cannot-use-example-grok-pattern-official-docs-in-grok-processor/358871/4 "2024-05-07T14:43:35Z")

</div>

I figured out the causes of the issue:

1. Need to configure output.elasticsearch in our filebeat-kubernetes.yaml to point to the ingest pipeline, by adding a `pipeline: our-pipeline` statement.
2. The YAML manifest file had `fields_under_root: true`. I didn't understand what that really meant until I reread the [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html). The custom field `tomcat_app_id` was being stored at the top level, not under a fields sub dictionary, so the conditional for the Grok processor was wrong.

Now the document is being processed as expected.

Thanks!!!