# Searching 1M document from 10M documents
**URL:** https://discuss.elastic.co/t/searching-1m-document-from-10m-documents/36602
**Category:** Elasticsearch
**Created:** [December 8, 2015, 5:59am UTC](https://discuss.elastic.co/t/searching-1m-document-from-10m-documents/36602 "2015-12-08T05:59:49Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@no\_jihun](https://discuss.elastic.co/u/no_jihun)
#### Post date: [December 8, 2015, 5:59am UTC](https://discuss.elastic.co/t/searching-1m-document-from-10m-documents/36602/1 "2015-12-08T05:59:49Z")
Hello.
I should like to have the benefit of your advice.
I wanna make a query to an index which have 10 million documents. each document's size is 1k~5k.
I expect the result document count is 10k in minimun and 1Million in maximum.
The query is quite simple. something like..
```auto
/myindex/mytype/_search
{
"query": {
"bool": {
"should": {
"query_string": {
"query": "baby car house star giant computer"
}
}
}
}
}
```
if some common keyword used It's hit count will increase(~1M)
(e.g query:"man morning car house go result")
if some specific keywrd used It's hit count will be small(1k~).
(e.g "offensive knife terror")
My final goal is get all of the each document's \_id which has hit.
(but I am not sure I will use min\_score)
In that case, if I set large fetch(about 1M) size to get the result at once ES will have OOM trouble.
On the other hands, if I set small fetch size, and use pagination to get the result ES will have deep pagination problem. [https://www.elastic.co/guide/en/elasticsearch/guide/current/pagination.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/pagination.html)
What will be the good way to get the large result set by query?
When I use SCAN it doesn't provide score value.
```auto
QueryBuilder queryBuilder = QueryBuilders.queryStringQuery(queryString);
SearchRequestBuilder builder = client.prepareSearch("usertext");
builder.setTypes("usertext");
builder.setQuery(queryBuilder);
builder.setSearchType(SearchType.SCAN);
builder.setSize(3000);
builder.setScroll(new TimeValue(1000));
SearchResponse response = builder.execute().actionGet();
int addCount = 0;
while (true) {
for (SearchHit hit : response.getHits()) {
addCount++;
uuidSet.add(hit.getId());
System.out.println(addCount + "," + uuidSet.size() + ", " + hit.getScore());
}
response = client.prepareSearchScroll(response.getScrollId()).setScroll(new meValue(10000)).execute().actionGet();
if (response.getHits().getHits().length == 0) {
break;
}
}
```
---
### Author:  [@nik9000](https://discuss.elastic.co/u/nik9000)
#### Post date: [December 8, 2015, 7:04pm UTC](https://discuss.elastic.co/t/searching-1m-document-from-10m-documents/36602/2 "2015-12-08T19:04:29Z")
> [@no\_jihun](#):
>
> What will be the good way to get the large result set by query?
[scroll](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html) doesn't require `scan`. It should work if you just remove:
```auto
builder.setSearchType(SearchType.SCAN);
```
It won't be as efficient, but it should work.
You may also want to think about Elasticsearch's aggregations. They were created to answer questions about a large result set without having to pull back the whole thing. They get to take advantage of the columnar storage format for doc\_values, making them quite efficient. They can be plugged together in lots of ways and you can write an Elasticsearch plugin to define more of them.
> [@no\_jihun](#):
>
> builder.setScroll(new TimeValue(1000));
I believe this time value is pretty small. You'll probably want to make it a bit higher. Usually I think of these timeouts in minutes. Once the scroll context has timed out its gone and you've have to restart the process.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 11:32pm UTC](https://discuss.elastic.co/t/searching-1m-document-from-10m-documents/36602/3 "2017-07-05T23:32:31Z")