Nginx logs on Kubernetes not parsed

Elastic Stack Beats
1

We are running Filebeat on Kubernetes to ingest nginx logs. Its going fine except for the fact that lines are not being parsed. The entire nginx log line shows up under "message" in elastic.

"message": "10.199.12.66 - - [08/Aug/2018:07:23:24 +0000] "POST /api/v0.1/elasticsearch/clusters-*/_search HTTP/1.1" 200 147857 "https://url/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"",

Configuration is minimalistic.

filebeat.yml

filebeat.autodiscover:
providers:
- type: kubernetes
hints.enabled: true
output:
elasticsearch:
...

nginx kubernetes manifest
metadata:
name: nginx
annotations:
co.elastic.logs/module: nginx
co.elastic.logs/fileset.stdout: access
co.elastic.logs/fileset.stderr: error

I have seen similar threads on the forum that has died after a while. Did anyone find a solution here?

/Mats

2

hello @iremmats,

I have a few questions:

  • Can you take a look at your Filebeat logs? I presume there is an ingest pipeline parsing error.
  • Also are you using the default nginx log format or a custom one?
  • What version of Filebeat you are running?

Thanks

3

@iremmats I think you are running into a bug, we have this issue open to track it.

4

Thats what I thought too. :slight_smile:

No errors in logs.
No custom log format.
Tested with 6.3.2 and 6.3.0.

I'll keep an eye on that issue.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.