It is my understanding that Logstash can handle multiple instances of filebeat. My question is whether there is a pro to configuring multiple Logstash instances to handle 3-4 filebeat instances (all on different servers).
Also, for now I have one filebeat instance sending to logstash on a different server. If I were to add another filebeat instance, would I need to restart logstash for it to recognize a new filebeat input?
For now I have one grok pattern specified for one filebeat instance. If I add another filebeat instance, I plan on adding a different grok filter.
So I will need to be modifying my configuration file (using if/else statements depending on log_type specified in filebeat.yml files). In that case, I assume I will have to restart the logstash instance right?
And if so, when I run the logstash instance again, filebeat will be able to correctly recognize where it left off (even with rotating logs)?
You do not need to restart, you can use SIGHUP to tell logstash to re-read the configuration, or enable --config.reload.automatic so that logstash polls to see if the configuration has changed. I do not think you will lose data when that happens but I am not absolutely certain.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.