Filebeat.template.json

[Arnaud_Domard]

Hi

I am looking for a template matching access .log file (apache log file).
I need to parse the message which is the raw apache log in kibana.
Where can i find the documentation to build my own filebeat template ?
Fields type accepted...

Thanks for your help

arnaud

[tudor (Tudor Golubenco)]

For the moment I would recommend using Logstash to parse the access logs, see an example here: https://www.elastic.co/guide/en/logstash/current/config-examples.html#_processing_apache_logs

This parsing can also be done with the Elasticsearch Ingest Node, and in the future we plan to make it possible for Filebeat to upload Ingest Node configurations. That part is not yet ready, though.

[nono1974]

Hi thank you for the reply.

I switch my apache log to json encoding.

I put this in my custom apache acces log

LogFormat "{ "index" : { "_index" : "Apachebeat-%{%Y.%m.%d}t", "_type" : "ApacheHttpRequestEvent" }}\n { "timestamp": "%{%Y-%m-%dT%H:%M:%S%z}t", "type": "ApacheHttpRequestEvent", "version": "2.4.25", "origin":{"host":{"name":"%h"},"software":{"name":"httpd","instance":"0","processId":"%{pid}P","threadId":"%{tid}P"},"module":{"executionId":"","name":"%U"}},"tags":["domain:technical","tier:software","severity:info"],"remoteAddr":"%a","forwardedFor":"%{X-Forwarded-For}i","referer":"%{Referer}i","method":"%m","protocol":"%H","parameters":"%q","contentType":"%{Content-Type}o","correlationId":"%{HTTP_CORRELATION_ID}e","transactionId":"%{X-TRANSACTIONID}i","userIdTech":"%{X-USERID-TECH}i","userIdTarget":"%{X-USERID-TARGET}i","status":%>s,"latency":%D,"size":%O}" combined_json

I don't see clearly what i should change in the filebeat.yml to digest this json nativaly to ES.

Could you please give some clues ?

regards

[ruflin]

Have a look at the filebeat json docs: https://www.elastic.co/guide/en/beats/filebeat/master/configuration-filebeat-options.html#config-json