# Search Optimization and FileDownload

**URL:** <https://discuss.elastic.co/t/search-optimization-and-filedownload/43347>\
**Category:** Elasticsearch\
**Created:** [March 3, 2016, 8:27am UTC](https://discuss.elastic.co/t/search-optimization-and-filedownload/43347 "2016-03-03T08:27:32Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![sureshadapa](https://avatars.discourse-cdn.com/v4/letter/s/0ea827/32.png) [@sureshadapa](https://discuss.elastic.co/u/sureshadapa)\
**Post date:** [March 3, 2016, 8:27am UTC](https://discuss.elastic.co/t/search-optimization-and-filedownload/43347/1 "2016-03-03T08:27:32Z")

</div>

I have data loaded in ES using logstash, and i am using elasticsearch.js in my App to query and fetch the data. I am looking for an optimum solution in which my search is quick and Data File-Size is reduced.

In the present set up search - took 647, - hits total=45806 and Data/FileSize: 14.1MB, browser download Time:6.85s, Which was originally - took 3153, - hits total=45806 and Data/FileSize: 31.9MB, browser download Time:14.77s

I have tried to optimize the JSON search request as below.I need suggestion if there is better one then Ver1.3. I guess the problem in my App is on client side with filedownload option where Data/FileSize is hugh.

Ver1.0  
GET k00125\_car/\_search  
{"query":{"filtered":{"query":{"query\_string":{"analyze\_wildcard":true,"query":""}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"gte":1286952143643}}}],"must\_not":[]}}}},"highlight":{"pre\_tags":["@kibana-highlighted-field@"],"post\_tags":["@/kibana-highlighted-field@"],"fields":{"":{}},"fragment\_size":2147483647},"size":1000000,"sort":[{"focus\_tier":{"order":"desc","unmapped\_type":"boolean"}}],"aggs":{"2":{"date\_histogram":{"field":"@timestamp","interval":"1M","pre\_zone":"+05:30","pre\_zone\_adjust\_large\_interval":true,"min\_doc\_count":0,"extended\_bounds":{"min":1286952143643,"max":1444718543643}}}},"fields":["\*","source"],"scriptfields":{},"fielddata\_fields":["@timestamp"]}

Ver1.1  
GET k00125\_car/\_search  
{  
"query": { "match\_all": {} },  
"size":1000000,  
"source": ["bunit","companycode","customer\_number","focus\_tier","name","contact\_phone","service\_address","sum\_svchrg"]  
}

Ver1.2  
GET k00125\_car/\_search  
{  
"size":1000000  
}

Ver1.3  
GET k00125\_car/\_search  
{  
"fields": ["bunit","company\_code","customer\_number","focus\_tier","name","contact\_phone","service\_address","sum\_svchrg"],  
"size":1000000

}

---

<div class="post-metadata">

**Author:** ![jimczi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jimczi/32/47985_2.png) [@jimczi](https://discuss.elastic.co/u/jimczi)\
**Post date:** [March 3, 2016, 9:47am UTC](https://discuss.elastic.co/t/search-optimization-and-filedownload/43347/2 "2016-03-03T09:47:13Z")

</div>

If you want to retrieve a lot of results (size: 1000000) you should use the scroll API:  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html)

---

<div class="post-metadata">

**Author:** ![sureshadapa](https://avatars.discourse-cdn.com/v4/letter/s/0ea827/32.png) [@sureshadapa](https://discuss.elastic.co/u/sureshadapa)\
**Post date:** [March 3, 2016, 10:44am UTC](https://discuss.elastic.co/t/search-optimization-and-filedownload/43347/3 "2016-03-03T10:44:07Z")

</div>

Thankyou.  
I am using combination of scroll and fields, because if i use { "sort": ["\_doc"] } i feel that amount of data returned in response is hugh. Or is using filter more usefull. 🙂

GET k00125\_car/\_search?scroll=1m  
{  
"fields": ["bunit","company\_code","customer\_number","focus\_tier","name","contact\_phone","service\_address","sum\_svchrg"],  
"size":1000000  
}

---

<div class="post-metadata">

**Author:** ![jimczi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jimczi/32/47985_2.png) [@jimczi](https://discuss.elastic.co/u/jimczi)\
**Post date:** [March 3, 2016, 10:51am UTC](https://discuss.elastic.co/t/search-optimization-and-filedownload/43347/4 "2016-03-03T10:51:35Z")

</div>

When using the scroll you should not use size or at least not a size of 1000000. The initial query will give you a scroll\_id which should be passed to the scroll API in order to retrieve the next batch of results so it's not a one shot query. Using { "sort": ["\_doc"] } does not change the amount of data returned, it is an optimization which makes the query faster.  
Please read carefully this part of the documentation:  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html#search-request-scroll](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html#search-request-scroll)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:11pm UTC](https://discuss.elastic.co/t/search-optimization-and-filedownload/43347/5 "2017-07-05T23:11:40Z")

</div>