How does connection pooling internally work with logstash http plugin

Elastic Stack Logstash
1

The default pool max for http output plugin is 50 connections . What happens if all the 50 connections to send out are occupied and there are more logs lines coming into the logstash but all the 50 out connections are still busy , will the logs be dropped after waiting sometime or is there an internal queuing system which waits for a connection to be free and sends the logs lines to one of the available 50 connections ?I am understanding the need to increase pool_max on http output plugin for logstash

2

There are queues between the filters and the outputs. If the output cannot accept more data then the data will queue, potentially stopping the filters if the queue fills. That will result in data queueing between the input and the filters. If that queue fills the input stop accepting data. See https://www.elastic.co/guide/en/logstash/current/execution-model.html

3

Starting with Elastic-search 5.3, there’s an HTTP setting called http.content_type.required . If this option is set to true , and you are using Log-stash 2.4 through 5.2, you need to update the Elastic-search output plugin to version 6.2.5 or higher.

with regards

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.