I am using a 5 node elasticsearch cluster with a ~120GB index.
We are having issues with slow queries that have several filters. After profiling our queries, I see that
IndexOrDocValuesQuery is taking about 3.2 seconds each. Each query has about 4 or 5 of these steps which cause it to take 20 seconds. What is weird is that the TermQuery which I would imagine is much more expensive takes 64 ms.
Where should we poke around to get more information as to what could be slowing these queries down?
26 GB RAM
12 GB Heap
5 Data Nodes, 2 Masters, 2 Clients. Masters and clients are much lower speced than nodes.
Here is the query: https://gist.github.com/wakawaka54/d4cad864d858efc41a0b2806ea0041b4
Here is the query profile result: https://gist.github.com/wakawaka54/6b52fd0eef2b8ead1589241e65c239fe