I am currently sending apache access logs from a remote server to my Logstash server by running FileBeat on the remote server. This is working like a charm.
I would like to also send other logs with different log content using FileBeats from the same remote server to the same Logstash server and parse those logs files separately. For example I want to create a custom field and separate index for the other logs that I want to send from the same remote server.
Is this possible? If so what is the syntax that I would need to put in my Logstash config file? Can I just use one config file or would I need multiple config files?
Is this possible? If so what is the syntax that I would need to put in my Logstash config file?
What one typically does is assign different types to different kind of logs (you can do that from Filebeat). Then you can use conditionals in your Logstash configuration to do different things with different types of logs.
Can I just use one config file or would I need multiple config files?
What one typically does is assign different types to different kind of logs (you can do that from Filebeat). Then you can use conditionals in your Logstash configuration to do different things with different types of logs.
Right, so I'm having a difficult time understanding the syntax that needs to be configured in the filebeat.yml file and my logstash config file "test3.conf".
Here is what I have in my filebeat.xml:
filebeat.prospectors:
input_type: log <--This needs to be "log" if I change it to "tools-message" then filebeat doesn't start
paths:
/var/log/*.log
fields: {log_type: tools-message}
Here is what I have in my "test3.conf" logstash config file:
This logstash config file does not create an index with name "tools-message". Instead it creates an index called "log_index". I want to able to have multiple different indices based on the type of log that logstash is parsing from filebeats.
I believe I figured it out. Not sure if its the right way, or an unusual way of doing it but basically this is what my configs looks like and yields me two distinct indices per log file type:
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.