Speed question between v5 and 7 and 8

Elastic Stack Elasticsearch
1

Hi,
We are facing a situation which we would like to resolve.
We have a server running Elastic version 5.6 (on Docker). We wanted to upgrade and tried 7.17, on the same network, on a machine which is identical and also on Docker.
Please note that we have also deployed one on Elastic Cloud.
We are facing an issue however, where version 5 is significantly faster in running any query we throw at it.
We have left them running for the same amount of time and thrown at them the same queries.
The only real difference we see is that the number of shards was set by default to 5, but we had to explicitly do so on version 7 and 8 (by applying an index template).
The other is the removal of doc types.

For the following query we are getting 'took' times that are:

Version 5 -> 400ms
Version 7 & 8 -> 1000ms

Please note that we are intentionally wanting to get a very large number of results back.
Here is the query. Any help would be really appreciated:


{"size": 10000, "query": {"bool": {"filter": {"bool": {"must": [{"terms" : { "site" : ["investor_user"] }}]}}, "must_not": [{"query_string": {"query": "Global,UAE,GEMs,Middle East,GCC,EMEA,MENA,United Arab Emirates", "default_field": "geo_exclude"}}], "must": [{"query_string": {"query": "Global,UAE,GEMs,Middle East,GCC,EMEA,MENA,United Arab Emirates", "default_field": "geo2"}}, {"query_string": {"query": "Any,Financials", "default_field": "sec2"}}, {"query_string": {"query": "Equity,Debt", "default_field": "assetClass2"}}, {"query_string": {"query": "*~", "default_operator": "AND"}}, {"range": {"liquidity": {"lte": 9000}}}, {"range": {"mcap_from": {"lte": 100000}}}, {"range": {"mcap_to": {"gte": 100000}}}]}}, "aggs": {"group_by_asset": {"terms": {"field": "assetClass"}}, "group_by_city": {"terms": {"field": "city", "size": 120}}, "group_by_sec": {"terms": {"field": "sec", "size": 20}}, "group_by_geo": {"terms": {"field": "geo", "size": 20}}}, "sort": {"cir.3532": {"order": "desc", "missing": "_last"}},"_source": {
              "includes": [
                  "company", "companyName", "city", "geo", "geo2", "sec", "sec2", "assetClass", "assetClass2", "city2",
                  "geo_exclude", "liquidity", "mcap_from", "mcap_to", "*.3532"
              ],
              "excludes": ["meetings2.*"]
          }}

--- Sample responses (Version 8)


{
    "took": 1017,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 6332,
            "relation": "eq"
        },
        "max_score": null,
        "hits": [
            {
                "_index": "essearchuserindex",
                "_id": "3796",
                "_score": null,
                "_source": {
                    "city2": "London",
                    "meetingsInst": {
                        "3532": "2020-03-30 00:00:00"
                    },
                    "city": "London",
                    "companyName": "UBS Asset Management",
                    "liquidity": 1.0,
                    "assetClass": "Debt",
                    "assetClass2": "Debt",
                    "cir": {
                        "3532": 89
                    },
                    "geo": "CIS, GCC, Russia, Turkey",
                    "sec": "Any",
                    "shares": {
                        "3532": 31242
                    },
                    "geo2": "CIS, GCC, Russia, Turkey",
                    "sec2": "Any",
                    "cir_explain": {
                        "3532": "Holds shares in peers, Is a shareholder, Met before, Met peer(s), Met peer(s), Met peer(s), Met peer(s), Met before, Geo focus, Sector interest"
                    },
                    "geo_exclude": "",
                    "mcap_to": 9.99999999E8,
                    "meetingsInst2": {
                        "3532": "2020-03-30 00:00:00"
                    },
                    "mcap_from": 0.001,
                    "company": "UBS Asset Management",
                    "meetings": {
                        "3532": "2019-06-18 00:00:00"
                    }
                },
                "sort": [
                    89
                ]
            },

-- Sameple response (Version 5)

    "took": 291,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 6356,
        "max_score": null,
        "hits": [
            {
                "_index": "essearchuserindex",
                "_type": "usertype",
                "_id": "23737",
                "_score": null,
                "_source": {
                    "city2": "London",
                    "city": "London",
                    "companyName": "JP Morgan & Chase",
                    "liquidity": 0,
                    "assetClass": "Equity",
                    "assetClass2": "Equity",
                    "geo": "GEMs",
                    "sec": "Any",
                    "geo2": "GEMs",
                    "sec2": "Any",
                    "geo_exclude": "",
                    "mcap_to": 500000000,
                    "mcap_from": 0,
                    "company": "JP Morgan & Chase"
                },
                "sort": [
                    -9223372036854775808
                ]
            },
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.