Java api client bulk processor locking up, both ES 5 and 2.3

Elastic Stack Elasticsearch
1

I'm running into some weird issues with indexing a lot of documents using the java bulk processor in async mode. It runs well but we've noticed it deadlocking. Essentially running a thread dump it appears its waiting for another thread to complete in order for the current thread to get access to the semaphore. From what I can observe the cluster never responds back and is forever in a locked stage.

In terms of our cluster it has a high number of aliases and mappings per index, which we are looking now to reduce and refactor. But in the meantime, I can't understand why we would observe deadlocks where the cluster never responds back with a timeout. The ES 2.3 cluster seems to be much more prone to this issue, since doesn't have master only/data only setup. The 5.X cluster seems ATM ok with a restart. Nothing in the logs jump out that an issue has occurred.

I guess I'm trying to pinpoint the root cause, such that I can determine it is indeed in fact we have too many aliases and mappings.

2

I suggest posting your code

3

Thanks,
I figured out the issue sort of. Basically when I was trying to do an alias exists check it would sometimes respond back and sometimes never respond back. Apparently the cluster never times out thus making the processing freeze due to the semaphore lock. I'm pretty sure I just have way too many aliases, so I'm readjusting my cluster to store the minimal amount of aliases and mappings. Originally I wanted to leverage types and aliases for multi-tenancy across organizations but I'm concluding there are real limits based off further reading. When I disabled the alias checks, the ingestion doesn't lock up.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.