Heavy Bulk inserts -Read & Write performance

Im using ES 7.14.0 with 16 data hot nodes and 2 warm nodes. When I'm doing heavy inserts, there is high thread pool write active count, and sometimes I get NONodeLeftException
I'm using ES-Hadoop Library with spark 3.1.0 on kuberenetes.

If i have made index refresh interval to -1 jobs are not having any issues. BUt I make it -1 documents will not be searchable.

Also tried refresh_interval with 600s to 100s but not much help, the probability of error reduces but jobs takes time.

How can i achieve heavy write & read from elasticsearch ?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.