More and more Nested query is progressively getting slower and slower

Elastic Stack Elasticsearch
1

We have a few millions of documents, each can have zero to hundred of nested objects (thankfully its just one level of nesting, the rest are just text fields).

When issuing a boolean should match query with one nested query for the nested objects, it took 3sec to return 30 results paginated from more than 10k results.

However, as we add more and more nested query, e.g.

"bool": {
  "should": [{
    "dis_max": {
      "queries": [{
        "nested": {...},
        "nested": {...},
        "nested": {...},

the time needed to return results is getting slower and slower to a point it takes over a minute to return for a query with 25 nested queries.

Here are some stats:
No nested query 1sec
1 nested query 3sec
5 nested queries 8sec
11 nested queries 15sec
19 nested queries 40sec
25 nested queries over a minute

Is more than 10 nested queries already considered too much for Elasticsearch?

What other things can we try to reduce the query time?

Is it too unrealistic to try to reduce the time for say 20 nested queries to return within 5sec?

Thank you

2

The more query clauses you ahve the more work for Elasticsearch, especially if it is wrapped in a should clause and there are no filters to reduce the number of documents that need to be evaluated. Having 25 match clauses sounds like a quite complex query, so it would be interesting to learn what this does and see if there are any alternatives. It would also be useful to know why you have gone with nested documents in the first place. Sometimes a flattened data model can perform better, but that is hard to judge without knowing the data and use-case.

Not necessarily - it depends on the use case and latency requirements.

Given the data you have provided I would say that likely is unrealistic with the current approach.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.