Query super slow whenever first time on an index

Similar query

first time: 19.301 s

second time: 0.314 s

index status after the first query

cluster status

You have 973.385.595 docs with 1.123.353.812.871 (1.1 Tb if I'm not mistaken) with only 2 data nodes?

That might explain that it takes a lot of time to read the data from disk the first time. On the second run, I guess that the FS Cache is playing its role.

What is the query you are running BTW?

Ho! I just saw that BTW:

"spins": "true"

Use SSD Drives!

{

  "size":500,

  "sort":[{"@timestamp":{"order":"desc","unmapped_type":"boolean"}}],

  "query":

    {"bool":

      {"must":[

        {"query_string":

          {"analyze_wildcard":true,

            "default_field":"message",

            "query":"event_code:5156 AND _type:logs AND exa_parser_name:q-5156 AND host:QRADAR_5156_HOST_NAME AND data_type:process-network AND dest_port:88"

          }

        }],

        "must_not":[]

      }

    }

}

Yes 1T data.

It works well on similar physical machines.

If you did not notice I updated my answer in the meantime.

Thanks,

  1. is there anyway to get the time spend on fetching from disk?
  2. I tried close the index and reopen the index, seems query is still fast. FS cache not clear after close the index?
  1. I don't know
  2. Why would it? The OS does not know that we are not going to access the files anymore.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.