Elasticsearch Search Threads not making sense

Elastic Stack Elasticsearch
1

So, I am seeing interesting metrics that don't really make much sense to me. Given I'm using the datadog integration that tells me these details, but they seem accurate with everything else so I have confidence in the numbers. Anyway, I am seeing a case where the number of active search threads by host doesn't exceed 2 threads, and yet I have the search thread queue increasing over 1. From my understanding the search thread queue is only there to act as a buffer so that large querying can simply slow down and not outright fail. Am I misunderstanding this in the documentation?

The queue_size allows to control the initial size of the queue of pending requests that have no threads to execute them

2

Furthermore, I was noticing mild lag in kibana which got me digging into this. Upon increasing that number to 30 threads per host my cluster seems to be performing much better on 2TB of data.

3

Your understanding is correct.

It's not that easy though, you may have faster searches but you are taking resources from other areas that adversely impact operations.

4

But then why would I never see the active search threads max out? Why would the queue get filled at all?

5

Basically, this implies the node is overloaded and you're asking too much of it.

6

I would agree if my active search threads ever grew about 2, when my CPU count is 8. (8*3)/2+1 = 13 Which is why i'm confused here.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.