Way to buffer logs in logstash

I have configured logstash on multiple AWS instances. These logstash instances receive thousand of logs and I want a way to wait for the entire trace of logs in logstash and take action on the batch instead of a single log event. I thought of using persistent queues to buffer these logs in logstash but since I am having multiple instances the queue will not be a good option.
Is there some way to buffer logs for some time in logstash and after that send the batches of logs that I want to send to Elastic search?

You mean like a stack trace?
What is sending the logs to Logstash?

Logstash forwarder is sending the logs to logstash. I want a way to buffer logs in logstash for some time and do batch checking on them. If entire batch satisfies my criterion then i need to pass entire batch of logs to elastic search otherwise drop the entire batch.

@warkolm hey can u help me regarding this ?

This is no longer supported, you should upgrade to Beats urgently.

Logstash is not really made for this, I am not sure what you can do.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.