Is BulkIngester (replacement of 'Bulk Processor') in elasticsearch java api thread safe?

Use case: I have multiple kafka listeners for various topics. Each topic Listener will run in multiple threads (using spring's 'ConcurrentKafkaListenerContainer'). Listeners will be performing Update/Insert operations on Elasticsearch via its Java client (newer low-level client). I realized that performing bulk operations is very performant and hence will be do that. Also found that the client library provides a very good utility called 'BulkIngester'(replacement of 'Bulk Processor' in previous versions) which is exactly what I needed.

Query: Since multiple thread will be adding update/insert operations to 'BulkIngester', I would like to understand if it is thread safe? As its user, do I need to take care of any synchronization?

Also, I did go through the 'BulkIngester' code and functionally understood how it is working. It also has code to perform synchronization but couldn't fully wrap my head around it.


Yes. You can use it the same way you were using the BulkProcessor.

Small usage example here:

AFAICS in the code, it's using an internal synchronisation so it's safe to use the same instance from multiple threads:

Thanks very much David for clarification.

Looks like the Java concurrency - Locks are being used, that are more flexible and sophisticated thread synchronization mechanism.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.