# Kibana Query Taking too long

**URL:** <https://discuss.elastic.co/t/kibana-query-taking-too-long/320865>\
**Category:** Kibana\
**Created:** [December 9, 2022, 8:27am UTC](https://discuss.elastic.co/t/kibana-query-taking-too-long/320865 "2022-12-09T08:27:17Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Arjun\_Meena](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arjun_meena/32/109270_2.png) [@Arjun\_Meena](https://discuss.elastic.co/u/Arjun_Meena)\
**Post date:** [December 9, 2022, 8:27am UTC](https://discuss.elastic.co/t/kibana-query-taking-too-long/320865/1 "2022-12-09T08:27:17Z")

</div>

We have a visualisation that uses the below given query to fetch the data.

```auto
GET commerce-access-console-logs-*/_search
{
  "aggs": {
    "3": {
      "filters": {
        "filters": {
          "LUX-SearchProducts": {
            "bool": {
              "must": [],
              "filter": [
                {
                  "bool": {
                    "should": [
                      {
                        "multi_match": {
                          "type": "phrase",
                          "query": "\\/marketplacewebservices\\/v2\\/lux\\/products\\/serpsearch",
                          "lenient": true
                        }
                      },
                      {
                        "multi_match": {
                          "type": "phrase",
                          "query": "\\/marketplacewebservices\\/v2\\/lux\\/products\\/searchProducts",
                          "lenient": true
                        }
                      }
                    ],
                    "minimum_should_match": 1
                  }
                }
              ],
              "should": [],
              "must_not": []
            }
          },
          "LUX-SearchProducts-Error": {
            "bool": {
              "must": [],
              "filter": [
                {
                  "bool": {
                    "filter": [
                      {
                        "bool": {
                          "should": [
                            {
                              "multi_match": {
                                "type": "phrase",
                                "query": "\\/marketplacewebservices\\/v2\\/lux\\/products\\/serpsearch",
                                "lenient": true
                              }
                            },
                            {
                              "multi_match": {
                                "type": "phrase",
                                "query": "\\/marketplacewebservices\\/v2\\/lux\\/products\\/searchProducts",
                                "lenient": true
                              }
                            }
                          ],
                          "minimum_should_match": 1
                        }
                      },
                      {
                        "bool": {
                          "must_not": {
                            "bool": {
                              "should": [
                                {
                                  "match": {
                                    "responseCode": "200"
                                  }
                                }
                              ],
                              "minimum_should_match": 1
                            }
                          }
                        }
                      }
                    ]
                  }
                }
              ],
              "should": [],
              "must_not": []
            }
          },
          "LUX-CartDetails": {
            "bool": {
              "must": [],
              "filter": [
                {
                  "bool": {
                    "filter": [
                      {
                        "multi_match": {
                          "type": "phrase",
                          "query": "\\/marketplacewebservices\\/v2\\/lux\\/",
                          "lenient": true
                        }
                      },
                      {
                        "query_string": {
                          "query": "*cartDetails*"
                        }
                      },
                      {
                        "bool": {
                          "must_not": {
                            "bool": {
                              "should": [
                                {
                                  "query_string": {
                                    "fields": [
                                      "request"
                                    ],
                                    "query": "*APItest@gmail\\.com*"
                                  }
                                }
                              ],
                              "minimum_should_match": 1
                            }
                          }
                        }
                      }
                    ]
                  }
                }
              ],
              "should": [],
              "must_not": []
            }
          },
          "LUX-CartDetails-Error": {
            "bool": {
              "must": [],
              "filter": [
                {
                  "bool": {
                    "filter": [
                      {
                        "multi_match": {
                          "type": "phrase",
                          "query": "\\/marketplacewebservices\\/v2\\/lux\\/",
                          "lenient": true
                        }
                      },
                      {
                        "query_string": {
                          "query": "*cartDetails*"
                        }
                      },
                      {
                        "bool": {
                          "must_not": {
                            "bool": {
                              "should": [
                                {
                                  "query_string": {
                                    "fields": [
                                      "request"
                                    ],
                                    "query": "*APItest@gmail\\.com*"
                                  }
                                }
                              ],
                              "minimum_should_match": 1
                            }
                          }
                        }
                      },
                      {
                        "bool": {
                          "must_not": {
                            "bool": {
                              "should": [
                                {
                                  "match": {
                                    "responseCode": "200"
                                  }
                                }
                              ],
                              "minimum_should_match": 1
                            }
                          }
                        }
                      }
                    ]
                  }
                }
              ],
              "should": [],
              "must_not": []
            }
          },
          "LUX-Payment Intent": {
            "bool": {
              "must": [],
              "filter": [
                {
                  "bool": {
                    "filter": [
                      {
                        "multi_match": {
                          "type": "phrase",
                          "query": "\\/marketplacewebservices\\/v2\\/lux\\/",
                          "lenient": true
                        }
                      },
                      {
                        "query_string": {
                          "query": "*collectPaymentOrder*"
                        }
                      }
                    ]
                  }
                }
              ],
              "should": [],
              "must_not": []
            }
          },
          "LUX-Payment Intent-Error": {
            "bool": {
              "must": [],
              "filter": [
                {
                  "bool": {
                    "filter": [
                      {
                        "multi_match": {
                          "type": "phrase",
                          "query": "\\/marketplacewebservices\\/v2\\/lux\\/",
                          "lenient": true
                        }
                      },
                      {
                        "query_string": {
                          "query": "*collectPaymentOrder*"
                        }
                      },
                      {
                        "bool": {
                          "must_not": {
                            "bool": {
                              "should": [
                                {
                                  "match": {
                                    "responseCode": "200"
                                  }
                                }
                              ],
                              "minimum_should_match": 1
                            }
                          }
                        }
                      }
                    ]
                  }
                }
              ],
              "should": [],
              "must_not": []
            }
          },
          "LUX-CustomerLogin": {
            "bool": {
              "must": [],
              "filter": [
                {
                  "bool": {
                    "filter": [
                      {
                        "multi_match": {
                          "type": "phrase",
                          "query": "\\/marketplacewebservices\\/v2\\/lux\\/",
                          "lenient": true
                        }
                      },
                      {
                        "query_string": {
                          "query": "*customerLogin*"
                        }
                      }
                    ]
                  }
                }
              ],
              "should": [],
              "must_not": []
            }
          },
          "LUX-CustomerLogin-Error": {
            "bool": {
              "must": [],
              "filter": [
                {
                  "bool": {
                    "filter": [
                      {
                        "multi_match": {
                          "type": "phrase",
                          "query": "\\/marketplacewebservices\\/v2\\/lux\\/",
                          "lenient": true
                        }
                      },
                      {
                        "query_string": {
                          "query": "*customerLogin*"
                        }
                      },
                      {
                        "bool": {
                          "must_not": {
                            "bool": {
                              "should": [
                                {
                                  "match": {
                                    "responseCode": "200"
                                  }
                                }
                              ],
                              "minimum_should_match": 1
                            }
                          }
                        }
                      }
                    ]
                  }
                }
              ],
              "should": [],
              "must_not": []
            }
          }
        }
      },
      "aggs": {
        "2": {
          "avg": {
            "field": "responseTimes"
          }
        }
      }
    }
  },
  "size": 0,
  "fields": [
    {
      "field": "@timestamp",
      "format": "date_time"
    }
  ],
  "script_fields": {},
  "stored_fields": [
    "*"
  ],
  "runtime_mappings": {},
  "_source": {
    "excludes": []
  },
  "query": {
    "bool": {
      "must": [],
      "filter": [
        {
          "range": {
            "@timestamp": {
              "format": "strict_date_optional_time",
              "gte": "2022-12-08T08:09:07.023Z",
              "lte": "2022-12-09T08:09:07.023Z"
            }
          }
        }
      ],
      "should": [],
      "must_not": []
    }
  }
}

```

But for some reason this query is taking a lot of time to execute, sometimes 10 mins and sometime 15 mins. The CPU utilization of Elasticsearch goes upto 60-70% when this query is being executed.

We have a single node cluster and for the given time frame that above given query is searching for, we have **151,180,367** logs in elasticsearch in the given index.

Our system specs are:  
m5.8xLarge EC2 instance with 32 CPU cores, 128 GB ram and a 7.2 TB gp3 volume attached.

Please also let me know what else details I need to share here.

---

<div class="post-metadata">

**Author:** ![Venkata\_Raja](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/venkata_raja/32/114264_2.png) [@Venkata\_Raja](https://discuss.elastic.co/u/Venkata_Raja)\
**Post date:** [December 9, 2022, 8:36am UTC](https://discuss.elastic.co/t/kibana-query-taking-too-long/320865/2 "2022-12-09T08:36:19Z")

</div>

Hi ,

How many indices are these query searching ?How many shards? shard size?

---

<div class="post-metadata">

**Author:** ![Arjun\_Meena](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arjun_meena/32/109270_2.png) [@Arjun\_Meena](https://discuss.elastic.co/u/Arjun_Meena)\
**Post date:** [December 9, 2022, 8:50am UTC](https://discuss.elastic.co/t/kibana-query-taking-too-long/320865/3 "2022-12-09T08:50:57Z")

</div>

![image](https://us1.discourse-cdn.com/elastic/original/3X/0/c/0c5d2be2b08a3f76814e3df8975c49e28638c0b4.jpeg)

We are storing 30 days worth of logs for this index pattern. A new index gets created everyday which gets a new name (depending on the current date). So there are total 30 indices.  
For each index we have 1 shard and their size in mentioned in the last column of the above table.

In the query we are looking for the logs ranging from 8 Dec 12:00 AM to 9 Dec 12:00 AM.

---

<div class="post-metadata">

**Author:** ![Venkata\_Raja](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/venkata_raja/32/114264_2.png) [@Venkata\_Raja](https://discuss.elastic.co/u/Venkata_Raja)\
**Post date:** [December 9, 2022, 9:24am UTC](https://discuss.elastic.co/t/kibana-query-taking-too-long/320865/4 "2022-12-09T09:24:37Z")

</div>

Can you check your multi match query , mentioning which fields to search will reduce query time . If you are not sure on which field to search , try to implement [copy\_to | Elasticsearch Guide [8.5] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/copy-to.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 6, 2023, 9:25am UTC](https://discuss.elastic.co/t/kibana-query-taking-too-long/320865/5 "2023-01-06T09:25:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
