Hi all,
I have been playing around with filebeat on my CentOS machine. I am trying to get filebeat on the host OS to send logs of my Apache)containers to Elasticsearch, whilst also extracting the fields by using the Apache2 module. For some reason I am not able to get this to work. I am able to extract the raw Docker json messages, or nothing at all.
So I'd like to achieve that this message is nicely split in to fields, as defined in Apache2 module.
"message": "{"log":"1.1.1.1 - - [19/Nov/2018:16:00:38 +0000] \"GET /somepage HTTP/1.1\" 200 427 \"https://URL.php\" \"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0\"\n","stream":"stdout","time":"2018-11-19T16:00:38.6229692Z"}",
I have tried various configs, but this seems to be the one I distilled from other topics:
filebeat.autodiscover:
providers:
- type: docker
templates:
- condition.contains:
docker.container.image: website
config:
- module: apache2
error:
paths:
- "/docker/containers/${data.docker.container.id}/*.log"
I am running filebeat 6.5.0 with docker 18.06.1-ce. Docker is running under /docker/, not in /var/lib/docker. Filebeat is not running in a container itself.