Filebeat nginx grok error

Elastic Stack Beats
1

We are getting a strange grok error using the nginx module in filebeat 5.5.2. The server config is:

/etc/filebeat/filebeat.yml:

filebeat:

nginx filebeat module

modules:

  • module: nginx
    access:
    enabled: true
    var.pipeline: with_plugins
    var.paths:
    • /var/log/opscode/nginx/access.log
      prospector:
      tags: ["nginxaccess","chefserver"]
      error:
      enabled: true
      var.paths:
    • /var/log/opscode/nginx/error.log
      prospector:
      tags: ["nginxerror","chefserver"]
      config_dir: /etc/filebeat/conf.d
      name: est-chef-frontend-node1.campus.ksu.edu
      fields: {
      ipaddress: "10.139.40.10"
      }

output.elasticsearch:
hosts: ["https://elasticsearch.ksu.edu:5002"]
template.enabled: true
template.path: "filebeat.template.json"
template.overwrite: false
index: "filebeat-%{+yyyy.MM.dd}"
username: "filebeat"
password:

When we get the messages into elasticsearch, they are being given an error:

{
"_index": "filebeat-2017.08.24",
"_type": "doc",
"_id": "AV4WV2nlBaN2Bc9kgIf8",
"_version": 1,
"_score": null,
"_source": {
"@timestamp": "2017-08-24T22:23:08.843Z",
"offset": 10752933,
"beat": {
"hostname": "est-chef-frontend-node1",
"name": "est-chef-frontend-node1",
"version": "5.5.2"
},
"input_type": "log",
"source": "/var/log/opscode/nginx/access.log",
"message": "129.130.254.176 - - [24/Aug/2017:17:23:07 -0500] "GET /organizations/omedev/data/ome_webserver_www_ksu_edu/ksre_feedtechnology HTTP/1.1" 200 "0.015" 262 "-" "Chef Client/12.6.0 (ruby-2.1.6-p336; ohai-8.8.1; x86_64-linux; +https://chef.io)" "127.0.0.1:8000" "200" "0.015" "12.6.0" "algorithm=sha1;version=1.0;" "ome-vm6.campus.ksu.edu" "2017-08-24T22:22:55Z" "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 1150",
"type": "log",
"error": "Provided Grok expressions do not match field value: [129.130.254.176 - - [24/Aug/2017:17:23:07 -0500] \"GET /organizations/omedev/data/ome_webserver_www_ksu_edu/ksre_feedtechnology HTTP/1.1\" 200 \"0.015\" 262 \"-\" \"Chef Client/12.6.0 (ruby-2.1.6-p336; ohai-8.8.1; x86_64-linux; +https://chef.io)\" \"127.0.0.1:8000\" \"200\" \"0.015\" \"12.6.0\" \"algorithm=sha1;version=1.0;\" \"ome-vm6.campus.ksu.edu\" \"2017-08-24T22:22:55Z\" \"2jmj7l5rSw0yVb/vlWAYkK/YBwk=\" 1150]",
"tags": [
"nginxaccess",
"chefserver"
]
},
"fields": {
"@timestamp": [
1503613388843
]
},
"highlight": {
"tags": [
"@kibana-highlighted-field@nginxaccess@/kibana-highlighted-field@"
]
},
"sort": [
1503613388843
]
}

As an example, here are a couple of lines from the nginx access file:

10.139.40.1 - - [24/Aug/2017:17:24:50 -0500] "GET /organizations/isodev/data/oracledbclusters/uach-d-uad-db HTTP/1.1" 200 "0.022" 237 "-" "Chef Client/12.8.1 (ruby-2.1.6-p336; ohai-8.11.1; x86_64-linux; +https://chef.io)" "127.0.0.1:8000" "200" "0.021" "12.8.1" "algorithm=sha1;version=1.1;" "hris-d-dhris-db1.campus.ksu.edu" "2017-08-24T22:24:37Z" "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 1145

10.139.40.1 - - [24/Aug/2017:17:24:50 -0500] "GET /organizations/isodev/data/oracledbclusters/ome-t-ometst-db HTTP/1.1" 200 "0.015" 249 "-" "Chef Client/12.20.3 (ruby-2.3.1-p112; ohai-8.23.0; x86_64-linux; +https://chef.io)" "127.0.0.1:8000" "200" "0.015" "12.20.3" "algorithm=sha1;version=1.1;" "iso-telc-p-db2.campus.ksu.edu" "2017-08-24T22:24:37Z" "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 1147

Our configuration is using elasticsearch 5.4.1 with filebeat 5.5.2. We were running the 5.4.1 version of filebeat, and had the same error (the hope was that an upgrade would fix the issue).

I open open to thoughts and suggestions.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.