Well I got everything running .. at least partially. I am able to get logs from filebeat -> logstash -> elastic -> kibana
I have to admit i haven't read all documentation - which is way to much for my short time. But I seem to have an understanding problem.
I am using logstash as an "connection point". It receives data on a port and pushes it to elasticsearch - I am aware that filebeat is able to push directly to elastic but if there is a untypical log file created by software i can create a regex pattern by using a logstash filter combined with an IF condition - that's the reason for using logstash?!
But with this configuration it seems that filebeat is pushing the logs as plain text - I've thought that it is using templates and is working for most default log files - apache, syslog etc.
@timestamp: October 25th 2017, 13:05:45.611 offset: 96,200 @version: 1 beatname: testing_server beat.hostname: debian beat.version: 5.6.3 input_type: log host:debian
source: /var/log/daemon.log message: Oct 25 06:05:43 debian apachectl: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
type: logtags: beats_input_codec_plain_applied_id: AV9TNZefwb4CR4YbNQRC_type: log_index: filebeat-2017.10.25_score: -
Should be the bold part be parsed and not be within the message ?
What am I overlooking or am I completely wrong ?
PS: I will create another Topic later but maybe it is a very short answer.
Can i change the order of the tags in kibana? I prefer that the "message" is in a single/last line - which would make it easier to read?