Is there a way to configure the batch size in the elastic search output plugin? I have an Elastic Search server at AWS on nodes with a request max size set to 10MB. If the output plugin sends more than 10MB in a request, then I would lose data. Based on the documentation the batches are capped at 2…

I believe the internal batch size used will be the maximum number of documents that will be sent to Elasticsearch as part off a single bulk request.

Discuss the Elastic Stack

Make batch size configurable in elastic search output plugin

Elastic Stack Logstash

Patrik_Iselind (Patrik Iselind) March 27, 2018, 10:00am 9

Ah, very good point! Thanks a lot Christian!

Topic		Replies	Views
Elasticsearch Output Plugin - Batching, How? Logstash	2	693	April 23, 2021
Logstash Bulk Indexing Logstash	7	5242	November 22, 2018
Batch size in Logstash Logstash	4	3242	May 3, 2022
Elasticsearch Output - Request size exceeded error Logstash	9	4943	July 6, 2017
Logstash Batch Size/Workers log message Logstash	14	12363	July 6, 2017

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

Make batch size configurable in elastic search output plugin

Related topics