Elasticsearch Output Plugin - Batching, How?

Elastic Stack Logstash
1

In the Logstash elastic search output documentation it mentions...

Batch Sizes: This plugin attempts to send batches of events to the [Elasticsearch Bulk API]

I can't find any information on how the batches are buffered, is it time based or document count based, when is it flushed etc?

I have taken a quick look at GitHub - logstash-plugins/logstash-output-elasticsearch and I can't see any code mentioning batches only comments.

2

My understanding is that when a batch of events exits the pipeline it will get passed to multi_receive, which immediately indexes them. It calls the mixin, which calls submit, which calls safe_bulk, which call the bulk method of the client. That is the code which does size based batching, in case a single batch from the pipeline is larger than 20 MB.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.