Does elastic search give users capability to create custom thread pool? If its a yes for above question, how do we utilize the custom pool to serve specific set of APIs?
My requirement is to have separate thread pool for single ingest and bulk ingest APIs. Can I create a custom thread pool and divert all the bulk requests to it and let write thread pool serve single ingests?
No, it doesn't.
Elasticsearch used to do this, but combined the thread pools in 6.0.
We are using bulk api for upgrade and single ingest for regular load. The requirement is to let single ingests not get affected by any upgrade running (even on the weekends). Since we cannot control the upgrades, bulk ingests are eating up all the write threads and even the queue is full. Is there any option available to distribute number of threads for bulk, and everything else? We are considering client throttling for bulk as on option too.
Env has 3 ES nodes. Each write thread pool is set to 32 threads and the default queue of 200.
Yes, that sounds like the best way to implement the kind of traffic shaping you need.
How does bulk API utilize active threads of write thread pool? I have an index with 1 shard and 1 replica. When I push , say 100 documents, using bulk API to this index, will it employ 1 thread of write thread pool to do the ingests?
I have raised this as a separate query too, but as this thread is already active, asking it here too.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.