I have 6 nodes, and some of them have a huge number of refresh in queue.
How can i get rid of them ?
I already change the refresh_interval from 10s to 60s, but i guess refresh in queue are now useless. is there a way to delete them ?
Thanks
What type of storage do you have? Local SSDs? Are all nodes the same specification? Do you have any indices on these nodes for which you perform a lot of updates and/or use a lot of nested documents?
Hello.
Figured it out. A huge number of Spark threads updating the same document with a refresh_interval set to 1sec... This was a misconfiguration where the elasticsearch document_id was supposed to be different. Therefore instead of creating a huge number of different douments in ES, the cluster tried to update only one.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.