BulkProcessor insertions not happening in our environment

Elastic Stack Elasticsearch
1

We are creating the BulkProcessor object as shown below. While some upserts are processed successfully, subsequent upserts are not being completed. This issue is occurring in a high-scale environment where the upserts requests are high in number. We are adding requests using bulkProcessor.add(request);. When a heap dump was taken, we can see the requests present in the dump.

Are there any known issues with this approach, or have similar problems been reported previously?

For reference, we are using Elasticsearch version 6.8.12.

bulkProcessor = BulkProcessor.builder(bulkConsumer, getBulkProcessListener())
    .setBulkActions(2000)
    .setBulkSize(
        new ByteSizeValue(100, ByteSizeUnit.MB))
    .setFlushInterval(5) // in seconds
    .setConcurrentRequests(2).build();
2

BulkProcessor had some race conditions that could lead to deadlock. This caused us to stop using BulkProcessor inside Elasticsearch. I don't know that that's what you're hitting (the output of kill -3 might show it), but I would stay away from BulkProcessor, and upgrade to a supported version as soon as possible.

3

Can you please let me know what is the supported version where the issues mentioned in BulkProcessor is fixed?

4

I think BulkProcessor was removed in 7.0. It's definitely gone by 8.0. The whole Java client has been replaced since 6.8.