We want to put in some network resilience in our setup. As of now the systems directly send logs to ES cluster. There is always a question of loss of logs when the network is down. With that in mind I was thinking of putting in Logstash in between.
I was thinking of leveraging the Persistent queue feature. But my understanding is that the data gets received, gets stored on disk and then shipped onwards. This is something which is not needed by me for most of time. It is only when the network is down that I want the data to be stored away on disk to be sent when the network is up.
In short: I do not want to store the data when it can be sent straight to the ES cluster.
Is there any way I achieve this by tweaking settings in logstash.yml?