# Grok filter not adding new fields

**URL:** <https://discuss.elastic.co/t/grok-filter-not-adding-new-fields/240825>\
**Category:** Logstash\
**Created:** [July 12, 2020, 8:26am UTC](https://discuss.elastic.co/t/grok-filter-not-adding-new-fields/240825 "2020-07-12T08:26:06Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![vamshisiddarth](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vamshisiddarth/32/82759_2.png) [@vamshisiddarth](https://discuss.elastic.co/u/vamshisiddarth)\
**Post date:** [July 12, 2020, 8:26am UTC](https://discuss.elastic.co/t/grok-filter-not-adding-new-fields/240825/1 "2020-07-12T08:26:06Z")

</div>

We have the below grok filter configured for our journlabeat. The same was deployed on our local for filebeat was working fine but isn't adding the new fields on journalbeat.

```auto
filter {

    grok {
      patterns_dir => ["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-patterns-core-4.1.2/patterns"]
      match => { "message" => [
        '%{IPV4:client_ip} - - \[%{HTTPDATE:date}\] "%{WORD:method} %{URIPATH:request} %{URIPROTO:protocol}\/[1-9].[0-9]" (%{NUMBER:status}|-) (%{NUMBER:bytes}|-) "(%{URI:url}|-)" %{QUOTEDSTRING:client}'
        ]
        break_on_match => false
        tag_on_failure => ["failed_match"]
      }
    }
}

```

We tried adding the mutate filter for adding new fields using below but it isn't fetching the value and is printing the scalar values itself (example: %{client\_ip}).

```auto
mutate {
        add_field => {
           "client_ip" => "%{client_ip}"
           "date" => "%{date}"
           "method" => "%{method}"
           "status" => "%{status}"
           "request" => "%{request}"
        }
      }

```

The log which we are trying to match is as below.

```auto
::ffff:172.65.205.3 - - [09/Jul/2020:11:32:52 +0000] "POST /v1-get--profile HTTP/1.1" 404 71 "https://mycompany.com/customer/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36"

```

Could someone let me know what exactly are we doing wrong. Thanks in Advance.

---

<div class="post-metadata">

**Author:** ![Rom1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rom1/32/49699_2.png) [@Rom1](https://discuss.elastic.co/u/Rom1)\
**Post date:** [August 6, 2020, 4:24pm UTC](https://discuss.elastic.co/t/grok-filter-not-adding-new-fields/240825/2 "2020-08-06T16:24:22Z")

</div>

Hi,

What not using this grok pattern?  
`HTTPD_COMBINEDLOG`  
found here:

> <https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/httpd>

---

<div class="post-metadata">

**Author:** ![Jenni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenni/32/29684_2.png) [@Jenni](https://discuss.elastic.co/u/Jenni)\
**Post date:** [August 6, 2020, 6:23pm UTC](https://discuss.elastic.co/t/grok-filter-not-adding-new-fields/240825/3 "2020-08-06T18:23:38Z")

</div>

The closing curly bracket for your `match` option should be moved two lines further to the top because right now it also wraps the next two options of your grok filter.

(And you can definitely delete the mutate filter. Right now it basically says: Create a field called X and fill it with the content of the field X. That will never be a helpful operation 🙂 )

Edit: Ooh. I had not realized that Rom1 had digged out such an old thread 🙃

---

<div class="post-metadata">

**Author:** ![vamshisiddarth](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vamshisiddarth/32/82759_2.png) [@vamshisiddarth](https://discuss.elastic.co/u/vamshisiddarth)\
**Post date:** [August 9, 2020, 1:26pm UTC](https://discuss.elastic.co/t/grok-filter-not-adding-new-fields/240825/4 "2020-08-09T13:26:02Z")

</div>

Thank you guys for the response. This was a while ago and is fixed. A bit embarrassing that I don't remember what was the scenario back then. But we updated our grok to below and everything is working fine now. Posting it here, if anyone else lands here searching for the same. 🙂

```auto
grok {
      patterns_dir => ["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-patterns-core-4.1.2/patterns"]
      match => { "message" => [
        '%{IPV4:client_ip} - - \[%{HTTPDATE:date}\] "%{WORD:method} %{URIPATH:request}(%{URIPARAM:uriparam}|) %{URIPROTO:protocol}\/[1-9].[0-9]" (%{NUMBER:status}|-) (%{NUMBER:bytes}|-) "(%{URI:url}|-)" %{QUOTEDSTRING:client}'
        ]
        break_on_match => false
        tag_on_failure => ["failed_match"]
      }
    }
    mutate {
      convert => {
        method => "string"
        status => "integer"
        url => "string"
      }
    }

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 6, 2020, 1:26pm UTC](https://discuss.elastic.co/t/grok-filter-not-adding-new-fields/240825/5 "2020-09-06T13:26:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.