Query with multiple AND and OR clauses in a filter context?

Elastic Stack Elasticsearch
1

Is it possible to perform a complex query in just filter context? In my case scoring does not matter at all. The documentation states that in a bool query filter and must_not are performed in a filter context.

So can I wrap a query in filter parameter and have it include bool elements that have a should property? Seems to work, but is this a recommended way?

A simplified case would be: All users that are over the age of 20 AND have glasses AND (are software developers OR data analysts) AND (have a degree in analytics OR engineering)

Here is an example query (completely unrelated to the sentence ^):

GET index/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "terms": {
            "someField": [
              "0",
              "1"
            ]
          }
        },
        {
          "terms": {
            "someField2": [
              "3",
              "4",
              "0"
            ]
          }
        },
        {
          "match_phrase_prefix": {
            "someField3": "Mich"
          }
        },
        {
          "bool": {
            "should": [
              {
                "prefix": {
                  "someField4": "Test"
                }
              },
              {
                "prefix": {
                  "someField5": "Test"
                }
              }
            ]
          }
        },
        {
          "range": {
            "CreationTime": {
              "gte": "2019-08-08",
              "lte": "2019-10-08"
            }
          }
        }
      ]
    }
  }
}

Before I used a bool query with a must parameter.

Thanks in advance for any help.

2

Hey,

that is possible in general. Just be aware of how the behaviour of the bool query changes, when you have a must clause, as then a should clause becomes optional and thus is not needed to decide if a document should be filtered or not. In your example it looks, as if you simply use the should clause as an OR combination between two queries, which makes sense to me.

hope this helps!

--Alex

3

Thanks, is this the most efficient way to do this? I am having some performance issues when performing parallel requests to an index

4

Try using the profile API to find out what is causing the performance issues. You may want to try with kibana as a start to have a nice UI around this, see https://www.elastic.co/guide/en/kibana/7.4/xpack-profiler.html

5

Thanks for the quick reply, nothing seems out of the ordinary when I run the queries from Kibana or navigate through the application manually, all queries are very fast in these cases. Slowness only happens when ~10 users run searches in parallel.

6

are you hitting a lot of shards with each query, so that the thread pools and queues become saturated quickly?

7

How can I investigate that? I've tested with 5 shards and 1 shard, that does not seem to matter as it's slow with both configurations.

Thanks for the help again :smile:

8

The node stats API shows information about searches being rejected or delayed because they are queued, that might be a first target for checking. See https://www.elastic.co/guide/en/elasticsearch/reference/7.4/cluster-nodes-stats.html

Also, are there any messages in the elasticsearch log file, that may help here?

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.