Nginx logs on Kubernetes not parsed

[iremmats (Mats Iremark)]

We are running Filebeat on Kubernetes to ingest nginx logs. Its going fine except for the fact that lines are not being parsed. The entire nginx log line shows up under "message" in elastic.

"message": "10.199.12.66 - - [08/Aug/2018:07:23:24 +0000] "POST /api/v0.1/elasticsearch/clusters-*/_search HTTP/1.1" 200 147857 "https://url/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"",

Configuration is minimalistic.

filebeat.yml

filebeat.autodiscover:
providers:
- type: kubernetes
hints.enabled: true
output:
elasticsearch:
...

nginx kubernetes manifest
metadata:
name: nginx
annotations:
co.elastic.logs/module: nginx
co.elastic.logs/fileset.stdout: access
co.elastic.logs/fileset.stderr: error

I have seen similar threads on the forum that has died after a while. Did anyone find a solution here?

/Mats

[pierhugues (Pier-Hugues Pellerin)]

hello @iremmats,

I have a few questions:

• Can you take a look at your Filebeat logs? I presume there is an ingest pipeline parsing error.
• Also are you using the default nginx log format or a custom one?
• What version of Filebeat you are running?

Thanks

[pierhugues (Pier-Hugues Pellerin)]

@iremmats I think you are running into a bug, we have this issue open to track it.

[iremmats (Mats Iremark)]

Thats what I thought too.

No errors in logs.
No custom log format.
Tested with 6.3.2 and 6.3.0.

I'll keep an eye on that issue.