I use BulkProcessor to index documents in java. And I want to set RefreshPolicy to each BulkRequest that processor created. But I can not find a method to do it. Is there a way to do so that I do not find? Or why is BulkProcessor not support setting RefreshPolicy?
You can use the Listener.afterBulk to send a refresh after a successful bulk indexing.
I think it has not been added, because we assume you keep writing a lot of data using this, which means, you dont usually refresh after a single bulk request, which you dont control exactly when that has been executed.
It is not possible to use RefreshPolicy in bulk because it is counterintuitive: bulk processing was designed to index a collection of documents in a single operation to avoid frequent refreshes. If each document would be refreshed within the bulk, the cluster would be flooded with segment processing and would collapse soon.
The correct method is to execute all the bulk indexing operations first, and after the last bulk operation, a single refresh should be executed.
Thanks. Finally, I add the RereshPolicy.WAIT_UNTIL to bulk request in the Listener.beforeBulk.
Thanks. I do not want to force refreshing the documents after every bulk. Instead, I want to wait until documents was refreshed.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.