BulkProcessor hangs instead of timeout

Elastic Stack Elasticsearch
1

Elasticsearch version: 6.1.3, 2 nodes
Java Rest High Level Client version: 6.4.0
Bulk size: ~10MB, flush every 5 seconds, concurrentRequests = 1
very similar issues, but without solution:

My app sometimes hangs when I use BulkProcessor. I checked in JConsole that it happens in BulkRequestHandler.execute(...) when it tries to acquire a semaphore. That semaphore is being released in request's callback - so I think the callback isn't triggered for previous request. I don't see any elastic's messages in log or any exceptions.

Simultaneously there is a Logstash instance writing to the same Elastic cluster. It shows up, that when my apps is having above problem, Logstash sometimes logs HTTP 429 errors. But, I thought that BulkProcessor can handle such situation and simply retry the request after waiting some time. In the worst case, I would expect a timeout. What can I do to deal with this?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.