# Elasticsearch query is too slow

**URL:** <https://discuss.elastic.co/t/elasticsearch-query-is-too-slow/233003>\
**Category:** Elasticsearch\
**Created:** [May 17, 2020, 4:14pm UTC](https://discuss.elastic.co/t/elasticsearch-query-is-too-slow/233003 "2020-05-17T16:14:45Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![niraj\_pandey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niraj_pandey/32/44034_2.png) [@niraj\_pandey](https://discuss.elastic.co/u/niraj_pandey)\
**Post date:** [May 17, 2020, 4:14pm UTC](https://discuss.elastic.co/t/elasticsearch-query-is-too-slow/233003/1 "2020-05-17T16:14:46Z")

</div>

Hi ,

I am using a python script to pull the data from elasticsearch and process it. It has around 5000k records , but it's more then 24 hrs and it's still running. What i can do to make it fast.

Here is the query I am using in python

```auto
def job_query_acct2(starttime, endtime,cluster,queue):
    starttime = str(starttime)
    endtime = str(endtime)
    tzone = "America/Los_Angeles"
    begq = '{ "_source": {"includes":["max_mem","max_no_procs"]},"query": {"bool": {"must": ['
    boolq2 = '{"bool": {"must": [{"match": {"cluster.keyword": ' + '"' + cluster + '"}},{ "match_phrase": { "queue": { "query":' + '"' + queue +'" } } } ,{ "range": { "event_time": {"lte":' + '"'+endtime + '", "gte":' + '"' + starttime + '", "time_zone": ' + '"'+tzone + '","format": "strict_date_optional_time" } } }]}}'
    endq = ']}} }'
    jobqry = begq + boolq2 + endq
    return jobqry

res = es.search(index=idx, scroll='2m', body=jobqry,timeout='1200s')

```

---

<div class="post-metadata">

**Author:** ![niraj\_pandey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niraj_pandey/32/44034_2.png) [@niraj\_pandey](https://discuss.elastic.co/u/niraj_pandey)\
**Post date:** [May 17, 2020, 4:15pm UTC](https://discuss.elastic.co/t/elasticsearch-query-is-too-slow/233003/2 "2020-05-17T16:15:22Z")

</div>

I am trying to get the data of one week (4-11 May)

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [May 18, 2020, 5:21pm UTC](https://discuss.elastic.co/t/elasticsearch-query-is-too-slow/233003/3 "2020-05-18T17:21:30Z")

</div>

Since you're just trying to pull out data with scrolls, I would move those clauses into the `filter` portion of the boolean query. This will remove the scoring aspect and should help speed it up. Right now the query is scoring all the boolean components which is not necessary because you don't actually care about the score.

Otherwise, larger scroll sizes can help (by reducing the number of round trips), and sorting the scroll by `_doc` will allow it to run faster (see the end of here: [https://www.elastic.co/guide/en/elasticsearch/reference/7.x/search-request-body.html#request-body-search-scroll](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/search-request-body.html#request-body-search-scroll))

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [May 18, 2020, 5:22pm UTC](https://discuss.elastic.co/t/elasticsearch-query-is-too-slow/233003/4 "2020-05-18T17:22:56Z")

</div>

I would also check and make sure there aren't any exceptions in the server log or client log 🙂 I've definitely run into slow batches before and realized afterwards that I had an error in my code and it was spewing exceptions for hours, not making any progress. This can happen if you're not using the scroll ID correctly for example

---

<div class="post-metadata">

**Author:** ![niraj\_pandey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niraj_pandey/32/44034_2.png) [@niraj\_pandey](https://discuss.elastic.co/u/niraj_pandey)\
**Post date:** [May 19, 2020, 6:10am UTC](https://discuss.elastic.co/t/elasticsearch-query-is-too-slow/233003/5 "2020-05-19T06:10:54Z")

</div>

I changed the below line and it's running very fast now. I am not sure if there any issue while using this any idea ?

> [@niraj\_pandey](#):
>
> begq = '{ "size": 10000, "\_source": {"includes":["max\_mem","max\_no\_procs"]},"query": {"bool": {"must": ['

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 16, 2020, 6:10am UTC](https://discuss.elastic.co/t/elasticsearch-query-is-too-slow/233003/6 "2020-06-16T06:10:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.