I have an elasticsearch cluster running with a handful of nodes and it periodically (as in every few minutes) falls down with a master_not_discovered_exception. While investigating I notice that when I check _cat/pending_tasks there are hundreds of pending tasks with source == "add_listener".
Example: 672140 387ms HIGH _add_listener_
I'm trying to find out what kind of requests result in an add_listener task so that I can track down the source of the requests. Any help would be greatly appreciated.
We are running 2.2.0, on Windows Server 2012 R2. There are 11 nodes in the cluster. They are all master and data nodes. Right now I'm not actively indexing any documents but there are about 120,000 documents already indexed.
It seems like long queue times are causing the ping requests to timeout, but I'm not sure why the pending tasks queue is long when I'm not indexing or querying anything.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.