It seems like with RabbitMQ you can enable acknowledgements on the input so the message only gets deleted from the server if it was successfully received. Is there any way to make it so acknowledgments are sent only if the output succeeded (e.g. message gets inserted into elasticsearch)?
My end goal is avoiding message loss if the logstash workers crash while processing messages.
Thanks!
No, this is currently not possible. However, Logstash's internal buffers are quite small (20 events, or maybe 40) so that's the greatest amount of events you'll ever lose.
20 input and 20 output.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.