Hi,
Is there a way to view the number of bulk requests waiting to be processed? It will help me to understand whether I have "threadpool.bulk.queue_size" set appropriately...
Regards,
David
Hi! Try this simple linux shell script:
watch -d -n 5 "curl -s 'localhost:9200/_cat/thread_pool?v&h=ip,port,ra,rq,fa,fq,sa,sq,gea,geq,ga,gq,maa,maq,fma,fmq,bs,br,ba,bq' | sort"
We don't recommend changing this for a number of reasons.
If you're threadpools are filling up then it's best to find out why, rather than just increase them.
Well, I've been seeing the beats input blocking on a congested logstash pipeline. Those events are associated with RemoteTransportExceptions in elasticsearch. In logstash, the elasticsearch output workers are set to 2 and flush_size is set to 500.
Other references I've seen to this type of log ingestion problem suggest raising the bulk.queue_size parameter...