Here is the result of
node_name name active queue rejected
node-5203 refresh 0 0 0
node-5202 refresh 10 13676 0
node-5206 refresh 0 0 0
node-5201 refresh 10 75341 0
node-5205 refresh 1 0 0
node-5204 refresh 10 3494 0
I have 6 nodes, and some of them have a huge number of refresh in queue.
How can i get rid of them ?
I already change the refresh_interval from 10s to 60s, but i guess refresh in queue are now useless. is there a way to delete them ?
Does the system load avg high in the nodes?
And How the io util grows to each node when this happen?
What type of storage do you have? Local SSDs? Are all nodes the same specification? Do you have any indices on these nodes for which you perform a lot of updates and/or use a lot of nested documents?
Figured it out. A huge number of Spark threads updating the same document with a refresh_interval set to 1sec... This was a misconfiguration where the elasticsearch document_id was supposed to be different. Therefore instead of creating a huge number of different douments in ES, the cluster tried to update only one.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.