# Custom nginx module log format error

**URL:** <https://discuss.elastic.co/t/custom-nginx-module-log-format-error/152791>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 17, 2018, 9:45am UTC](https://discuss.elastic.co/t/custom-nginx-module-log-format-error/152791 "2018-10-17T09:45:13Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![damonops](https://avatars.discourse-cdn.com/v4/letter/d/c5a1d2/32.png) [@damonops](https://discuss.elastic.co/u/damonops)\
**Post date:** [October 17, 2018, 9:45am UTC](https://discuss.elastic.co/t/custom-nginx-module-log-format-error/152791/1 "2018-10-17T09:45:13Z")

</div>

my filebeat nginx log format:

```auto
  "\"?%{IP_LIST:nginx.access.remote_ip_list} - %{DATA:nginx.access.user_name} \\[%{HTTPDATE:nginx.access.time}\\] \"%{GREEDYDATA:nginx.access.info}\" %{NUMBER:nginx.access.response_code} %{NUMBER:nginx.access.body_sent.bytes} \"%{DATA:nginx.access.referrer}\" \"%{DATA:nginx.access.agent}\" \"%{DATA:nginx.access.x_forwarded}\" \"%{DATA:nginx.access.request_time}\" \"%{DATA:nginx.access.upstream_addr}\" %{NUMBER:nginx.access.upstream_status} \"%{DATA:nginx.access.upstream_response_time}\" %{NUMBER:nginx.access.upstream_cache_status}"

```

my nginx log :

```auto
219.148.158.41 - - [17/Oct/2018:16:11:05 +0800] "POST /userapi/users/login/signin HTTP/1.1" 400 65 "https://monitoring-beta.cloudwise.com/users/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36" "-" "0.007" "192.168.2.10:8002" 400 "0.007" -

```

But I get a error.message from kibana:

```auto
Provided Grok expressions do not match field value: [219.148.158.41 - - [17/Oct/2018:16:11:05 +0800] \"POST /userapi/users/login/signin HTTP/1.1\" 400 65 \"https://monitoring-beta.cloudwise.com/users/login\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36\" \"-\" \"0.007\" \"192.168.2.10:8002\" 400 \"0.007\" -]

```

So please help me and Thanks.

---

<div class="post-metadata">

**Author:** ![Mario\_Castro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario_castro/32/35107_2.png) [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)\
**Post date:** [October 17, 2018, 10:59am UTC](https://discuss.elastic.co/t/custom-nginx-module-log-format-error/152791/2 "2018-10-17T10:59:51Z")

</div>

Hi @damonops 🙂

I don't see any matching expression that uses `IP_LIST`. Maybe you want to use `IP` instead?

Also, as the message implies, your Grok pattern doesn't match your log line. For example, your pattern looks for an IP and a User name but your log starts with an IP (good), some double dash and a date (no user here)

You can test Grok patterns here: [http://grokconstructor.appspot.com/do/match](http://grokconstructor.appspot.com/do/match)

---

<div class="post-metadata">

**Author:** ![damonops](https://avatars.discourse-cdn.com/v4/letter/d/c5a1d2/32.png) [@damonops](https://discuss.elastic.co/u/damonops)\
**Post date:** [October 18, 2018, 3:52am UTC](https://discuss.elastic.co/t/custom-nginx-module-log-format-error/152791/3 "2018-10-18T03:52:55Z")

</div>

Thank you，sorry,Less content is provided, and more info

Default nginx module log format:

```auto
"grok": {
      "field": "message",
      "patterns":[
        "\"?%{IP_LIST:nginx.access.remote_ip_list} - %{DATA:nginx.access.user_name} \\[%{HTTPDATE:nginx.access.time}\\] \"%{GREEDYDATA:nginx.access.info}\" %{NUMBER:nginx.access.response_code} %{NUMBER:nginx.access.body_sent.bytes} \"%{DATA:nginx.access.referrer}\" \"%{DATA:nginx.access.agent}\""
        ],
      "pattern_definitions": {
        "IP_LIST": "%{IP}(\"?,?\\s*%{IP})*"
      },
      "ignore_missing": true
    }
  }

```

I just add some fileds,like:

```auto
"grok": {
      "field": "message",
      "patterns":[
        "\"?%{IP_LIST:nginx.access.remote_ip_list} - %{DATA:nginx.access.user_name} \\[%{HTTPDATE:nginx.access.time}\\] \"%{GREEDYDATA:nginx.access.info}\" %{NUMBER:nginx.access.response_code} %{NUMBER:nginx.access.body_sent.bytes} \"%{DATA:nginx.access.referrer}\" \"%{DATA:nginx.access.agent}\" \"%{DATA:nginx.access.x_forwarded}\" \"%{DATA:nginx.access.request_time}\" \"%{DATA:nginx.access.upstream_addr}\" %{NUMBER:nginx.access.upstream_status} \"%{DATA:nginx.access.upstream_response_time}\" %{NUMBER:nginx.access.upstream_cache_status}"
        ],
      "pattern_definitions": {
        "IP_LIST": "%{IP}(\"?,?\\s*%{IP})*"
      },
      "ignore_missing": true
    }
  }

```

See other information in default file:

```auto
/usr/share/filebeat/module/nginx/access/ingest/default.json

```

Oh, my filebeat version is 6.3.0,thanks

---

<div class="post-metadata">

**Author:** ![damonops](https://avatars.discourse-cdn.com/v4/letter/d/c5a1d2/32.png) [@damonops](https://discuss.elastic.co/u/damonops)\
**Post date:** [October 19, 2018, 10:33am UTC](https://discuss.elastic.co/t/custom-nginx-module-log-format-error/152791/4 "2018-10-19T10:33:27Z")

</div>

I hope there is a example in the documents about customing log module format.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 16, 2018, 10:33am UTC](https://discuss.elastic.co/t/custom-nginx-module-log-format-error/152791/5 "2018-11-16T10:33:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.