Filter vs bool Query Performance , getting bad performance on using Filters

Elastic Stack Elasticsearch
1

Hi,
Generally it is recommended to use filters for optimal performance, but I am getting 5 times more latency when using filters compared to bool query.
However for me it does not make sense to use bool as logically I inly have to filter.

The queries are

GET entity/entity/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "must": [
              {
                "terms": {
                  "type": [
                    "tags"
                  ]
                }
              },
              {
                "terms": {
                  "state": [
                   0
                  ]
                }
              }
            ]
          }
        },
        {
          "bool": {
            "should": [
              {
                "prefix": {
                  "name": {
                    "value": "a"
                  }
                }
              },
              {
                "prefix": {
                  "ID": {
                    "value": "a"
                  }
                }
              }
              
            ]
          }
        }
      ]
    }
  }
}

The other query is

GET entity/entity/_search
{
  "query": { 
    "bool": { 
      "should": [
              {
                "prefix": {
                  "name": {
                    "value": "a"
                  }
                }
              },
              {"prefix": {
                  "ID": {
                    "value": "a"
                  }
              }
              }
              
            ],
      "filter": [ 
        { "term":  { "type": "tags" }},
        {"term":{"state":0}}
      ]
    }
  }
}

The filter query is comes out in 3 seconds and bool query takes 300-400 ms.
The total number of hits differ significantly with a 10K on filter query and 500 on bool query.
Total documents on entity index are 150 million.

Why does filter query have more number of hits then bool query and bad performance ?

2

Is that always the case? I mean after some runs it should be fast, right?

3

@dadoonet It is always the case.

4

Can you share the outputs of each query?

Which versions?

5

Your queries are not equivalent. The prefix queries are required to match in the first query but optional in the second query. This is why you are getting different counts. You need to put your SHOULD clauses under a MUST clause like in the first query.

6

@jpountz Both queries have should only.
The part which is in must in first is on must filter in other.

7

Take your first query and only replace must with filter. You will get the same number of hits. Your first query requires that one of the prefix queries matches while the second one doesn't. This is why you don't get the same numbers of hits and worse performance.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.