Elastic Search First Query is always slow and shards not getting distributed properly

I am creating per day index in Elastic Search ( version: 7.5.1 ). I have 3 nodes in total and 2 shards with one replica each, total disk: 5.6 TB and JVM Heap: 95.8 GB

per day index size is 40 GB with 110m documents.

I am facing a problem that whenever I query on elastic it is taking more than 15 seconds to give response. My first query takes around 12-18 seconds while second query takes only 1-2 seconds and sometimes milliseconds. what could be the exact reason for this behaviour?

secondly, all my shards are getting aligned to single node like some day Node 1 has 2 primary shards or other day Node 3 or Node 2 has. Shards are not getting allocated to different nodes and replica is never assigned. what could be fix here?

Reference query ( email type is keyword )

GET /today_index_20230327/_search
{
    "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "email": {
              "value": "abc@gmail.com"
            }
          }
        }
      ]
    }
  }
}

Elasticsearch 7.5.1 is quite old and has been EOL quite a while. I would recommend upgrading to at least 7.17. What is the specification of the cluster with respect to CPU, RAM and type of storage used?

What is the full output of the cluster stats API?

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.