Send multiple smaller bulk requests, e.g. a few thousand records each. A good guideline is to try and keep the size of the bulk request below 5MB in size. This approach also has the benefit of putting your cluster under less pressure.
Breaking data into multiple small request is taking time. Means I have 1000000 records and I am breaking data into 10 request. Per request I am sending 100000 records to elasticsearch. But It is taking time to create new json objects and sending one after another.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.