Grok from nginx-module gets error

I have nginx-module enabled in filebeat.yml:

- module: nginx
    var.paths: ["/testvar/nginx/access.log"]
    var.paths: ["/testvar/nginx/error.log"]

  hosts: ["localhost:9200"]

  host: "localhost:5601"

And all logs like: - - [02/Aug/2018:11:57:45 +0300] "" "POST /smth" 

are parsed in ElasticSearch. But with logs like:

-- - [02/Aug/2018:11:57:45 +0300] "" "POST /smth"

Kibana gets filed message.error with entry like:

Provided Grok expressions do not match field value:[...]

Any thoughts what I can do to improve my nginx-module?

Thanks @g.myznikov.tinkoff

I have added the above line in our integration test suite and I can reproduce the error:

AssertionError: not error expected but got: {u'beat': {u'hostname': u'sashimi', u'name': u'sashimi', u'version': u'7.0.0-alpha1'}, u'@timestamp': u'2018-08-08T14:20:02.157Z', u'offset': 2346, u'fileset': {u'name': u'access', u'module': u'nginx'}, u'source': u'/Users/ph/go/src/', u'host': {u'name': u'sashimi'}, u'error': {u'message': u'Provided Grok expressions do not match field value: [-- - [02/Aug/2018:11:57:45 +0300] \\"\\" \\"POST /smth\\"]'}, u'input': {u'type': u'log'}, u'message': u'-- - [02/Aug/2018:11:57:45 +0300] "" "POST /smth"', u'prospector': {u'type': u'log'}}

It look like that the following grok pattern is stricter than some real world use case.

 "pattern_definitions": {
        "IP_LIST": "%{IP}(\"?,?\\s*%{IP})*"

Can you tell me what is the version of nginx you are running? Also are you using the default log format for nginx or a custom one?

I use nginx version 1.12.1. I have not changed log format so it is probably default.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.