When logstash is not storing the logs it receive, what is present in pipeline of logstah?

Elastic Stack Logstash
1

When logstash is not storing the logs it receive, what is present in pipeline of logstah?

I am just trying to reload the pipeline (or delete some entries) when the pipeline is full or blocked, due to the unavailability of log receivers like lumberjack, and Elasticsearch.

Is there a way to find the pipeline size and if it reaches the threshold of 90%, can i empty it fully?

2

logstash has an at-least-once delivery model. If back-pressure from an output causes the pipeline to stall then it will stop processing events. The only way to empty the pipeline is for the output to accept the events and discard them due to a failure.

3

Yeah right,
In my case i am having one pipeline that process events, for two log receivers like lumerjack and elasticsearch, if my lumerjack goes down in few min, logstash stop processing the events to my Elasticsearch as well, and the pipeline is getting blocked.

I have to ensure at least one of my log receivers should get the logs processed from logstash, though other one is failed or not available for some reasons.

is there possibility?

4

See this thread for a possible solution.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.