I have a bunch of filebeat servers pointing to the logstash cluster via a service discovery endpoint (via consul). When I bounce the elasticsearch cluster, I usually start by stopping the filebeats, then the logstash agents, then elasticsearch; then I start elasticsearch, logstash, filebeat.
I won't be able to script the shutdown of filebeat (via service shutdown filebeat), but nevertheless I want to be able to pause it's prospecting while I bring the logstash/elasticsearch down and up.
Is there a way for me to have logstash send a "pause" response to filebeat? I don't want any data to be lost.
Beats will automatically try to reconnect if the remote becomes unavailable. As you use consul the Logstash hostnames won't change, so you are good to go. When using logstash set output.logstash.max_retries: -1. This enables infinite retry for all events. Filebeat/Winlogbeat already use infinite retry by default, no matter the max_retries setting (can not be disabled).
Why filebeat is probing for logstash to become available, the internal queues will fill up and create back-pressure. No data is collected until logstash becomes available again.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.