Query slower with ES 5 compared with ES 7

Elastic Stack Elasticsearch
1

Hi experts!

We recently migrated from a ES 5 to a 7 version.

We observed that some of our original queries became slower than before.

An example of query :

GET /[index_name]/_search?search_type=dfs_query_then_fetch
{
  "size": 100,
  "sort": [
    {
      "_score": {}
    },
    {
      "rs.raw": {
        "order": "asc"
      }
    },
    {
      "_script": {
        "order": "desc",
        "type": "number",
        "script": {
          "params": {
            "scores": {
              "actif": 3,
              "demenage": 2,
              "cesse": 1
            }
          },
          "lang": "painless",
          "source": "if(params.scores.containsKey(doc['state'].value)) { return params.scores[doc['state'].value];} return 0;"
        }
      }
    },
    {
      "siege": {
        "order": "desc"
      }
    }
  ],
  "_source": {
    "includes": [
      "field1",
	  "field5"
    ]
  },
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "must": [
              {
                "bool": {
                  "must": [
                    {
                      "bool": {
                        "should": [
                          {
                            "multi_match": {
                              "boost": 256,
                              "query": "TO_SEARCH",
                              "fuzziness": 0,
                              "operator": "and",
                              "fields": [
                                "rs.raw^3",
                                "l1_nomen.raw",
                                "sigle.raw^2",
                                "l2_comp.raw",
                                "enseigne.raw"
                              ]
                            }
                          },
                          {
                            "multi_match": {
                              "boost": 64,
                              "query": "TO_SEARCH",
                              "fuzziness": 0,
                              "operator": "and",
                              "fields": [
                                "rs.keyword^3",
                                "l1_nomen.keyword",
                                "sigle.keyword^2",
                                "l2_comp.raw",
                                "enseigne.keyword"
                              ]
                            }
                          },
                          {
                            "multi_match": {
                              "boost": 16,
                              "query": "TO_SEARCH",
                              "fuzziness": 0,
                              "operator": "and",
                              "fields": [
                                "rs.exact^3",
                                "l1_nomen.exact",
                                "sigle.exact^2",
                                "l2_comp.exact",
                                "enseigne.exact"
                              ]
                            }
                          },
                          {
                            "multi_match": {
                              "boost": 4,
                              "query": "TO_SEARCH",
                              "fuzziness": 1,
                              "operator": "and",
                              "fields": [
                                "rs.raw^3",
                                "l1_nomen.raw",
                                "sigle.raw^2",
                                "l2_comp.raw",
                                "enseigne.raw"
                              ]
                            }
                          },
                          {
                            "multi_match": {
                              "boost": 1,
                              "query": "TO_SEARCH",
                              "fuzziness": 0,
                              "operator": "and",
                              "fields": [
                                "rs^3",
                                "l1_nomen",
                                "sigle^2",
                                "l2_comp",
                                "enseigne"
                              ]
                            }
                          },
                          {
                            "query_string": {
                              "boost": 0,
                              "query": "TO_SEARCH*",
                              "default_field": "rs.exact",
                              "default_operator": "and",
                              "analyze_wildcard": false
                            }
                          }
                        ]
                      }
                    }
                  ]
                }
              }
            ]
          }
        }
      ]
    }
  }
}

Would you know why this happen?

Best regards,

2

Bonjour Vincent :wink:

Could you share what is the output of the first 10 lines of the response in both cases?

Some questions/thoughts:

  • Use 8.11.3 instead a very old 7.x version.
  • Does it still happen after some runs or just the first run is slow?
  • Could you try to isolate what is the costly part in your query?
  • You have many layers of non needed bool.must subqueries. Could you simplify that?
  • The last part of the query seems to be useless as you are boosting by 0. May be remove it?
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.