# Curl XGET not returning all the documents

**URL:** <https://discuss.elastic.co/t/curl-xget-not-returning-all-the-documents/95637>\
**Category:** Elasticsearch\
**Created:** [August 3, 2017, 3:59am UTC](https://discuss.elastic.co/t/curl-xget-not-returning-all-the-documents/95637 "2017-08-03T03:59:40Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![aliyesami](https://avatars.discourse-cdn.com/v4/letter/a/ecb155/32.png) [@aliyesami](https://discuss.elastic.co/u/aliyesami)\
**Post date:** [August 3, 2017, 3:59am UTC](https://discuss.elastic.co/t/curl-xget-not-returning-all-the-documents/95637/1 "2017-08-03T03:59:40Z")

</div>

I have 20 million documents in an index , the \_stats also confirm that and there are 914 distinct toll\_amt\_full field ( confirmed from hive table) .  
But curl aggregation query is only returning 10 documents ?

> hive\> select toll\_amt\_full,count(\*) from pa\_lane\_txn\_es group by toll\_amt\_full;  
> Query ID = root\_20170802225401\_c1f1147b-79b4-48b9-a2e6-e0bc4661737d  
> Total jobs = 1  
> Launching Job 1 out of 1
> 
> 28.0 57968  
> 29.0 1  
> .  
> .  
> .  
> 56100.0 1  
> Time taken: 1547.462 seconds, Fetched: **914 row(s**)  
> hive\>

> [elasticsearch@hadoop5 config]$ curl hadoop5:9200/lanetxn/\_stats?pretty  
> {  
> "\_shards" : {  
> "total" : 10,  
> "successful" : 10,  
> "failed" : 0  
> },  
> "\_all" : {  
> "primaries" : {  
> "docs" : {  
> " **count" : 20000020** ,  
> "deleted" : 0  
> },

> [elasticsearch@hadoop5 config]$ curl -XPOST 'hadoop5:9200/lanetxn/\_search?pretty' -d '  
> {  
> "size": 0,  
> "aggs": {  
> "group\_by\_toll\_amt\_full": {  
> "terms": {  
> "field": "toll\_amt\_full"  
> }  
> }  
> }  
> }'  
> {  
> "took" : 348,  
> "timed\_out" : false,  
> "\_shards" : {  
> "total" : 5,  
> "successful" : 5,  
> "failed" : 0  
> },  
> "hits" : {  
> "total" : 20000020,  
> "max\_score" : 0.0,  
> "hits" :   
> },  
> "aggregations" : {  
> "group\_by\_toll\_amt\_full" : {  
> "doc\_count\_error\_upper\_bound" : 96654,  
> "sum\_other\_doc\_count" : 6107485,  
> "buckets" : [  
> {  
> "key" : 132.0,  
> "doc\_count" : 2797710  
> },  
> {  
> "key" : 50.0,  
> "doc\_count" : 1843111  
> },  
> {  
> "key" : 140.0,  
> "doc\_count" : 1831116  
> },  
> {  
> "key" : 0.0,  
> "doc\_count" : 1354803  
> },  
> {  
> "key" : 100.0,  
> "doc\_count" : 1257437  
> },  
> {  
> "key" : 125.0,  
> "doc\_count" : 1173364  
> },  
> {  
> "key" : 82.0,  
> "doc\_count" : 1103776  
> },  
> {  
> "key" : 79.0,  
> "doc\_count" : 987945  
> },  
> {  
> "key" : 75.0,  
> "doc\_count" : 775867  
> },  
> {  
> "key" : 70.0,  
> "doc\_count" : 767406  
> }  
> ]  
> }  
> }  
> }

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 3, 2017, 4:00am UTC](https://discuss.elastic.co/t/curl-xget-not-returning-all-the-documents/95637/2 "2017-08-03T04:00:37Z")

</div>

Have a read of [https://www.elastic.co/guide/en/elasticsearch/reference/5.5/search-request-from-size.html](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/search-request-from-size.html)

---

<div class="post-metadata">

**Author:** ![aliyesami](https://avatars.discourse-cdn.com/v4/letter/a/ecb155/32.png) [@aliyesami](https://discuss.elastic.co/u/aliyesami)\
**Post date:** [August 3, 2017, 4:07am UTC](https://discuss.elastic.co/t/curl-xget-not-returning-all-the-documents/95637/3 "2017-08-03T04:07:04Z")

</div>

maybe I am not using the right syntax , added the from: clause but still getting 10 docs

> curl -XPOST 'hadoop5:9200/lanetxn/\_search?pretty' -d '  
> {  
> **"from" : 0, "size" : 1000,**  
> "size": 0,  
> "aggs": {  
> "group\_by\_toll\_amt\_full": {  
> "terms": {  
> "field": "toll\_amt\_full"  
> }  
> }  
> }  
> }'

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [August 3, 2017, 5:27am UTC](https://discuss.elastic.co/t/curl-xget-not-returning-all-the-documents/95637/4 "2017-08-03T05:27:47Z")

</div>

Move the size:1000 inside the terms agg

---

<div class="post-metadata">

**Author:** ![aliyesami](https://avatars.discourse-cdn.com/v4/letter/a/ecb155/32.png) [@aliyesami](https://discuss.elastic.co/u/aliyesami)\
**Post date:** [August 3, 2017, 7:40pm UTC](https://discuss.elastic.co/t/curl-xget-not-returning-all-the-documents/95637/5 "2017-08-03T19:40:49Z")

</div>

moved it inside but still not good ..

> curl -XPOST 'hadoop5:9200/lanetxn/\_search?pretty' -d '  
> {  
> "size" : 0,  
> "aggs" : {  
> "from" : 0, "size" : 1000,  
> "txn\_over\_time" : {  
> "date\_histogram" : {  
> "field" : "txn\_process\_date",  
> "interval" : "month"  
> }  
> }  
> }  
> }  
> '

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [August 3, 2017, 8:41pm UTC](https://discuss.elastic.co/t/curl-xget-not-returning-all-the-documents/95637/6 "2017-08-03T20:41:46Z")

</div>

You changed the aggregation. Was terms and now date\_histogram.  
You didn't move what I said where I said.

Is that a new problem you want to solve?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 31, 2017, 8:41pm UTC](https://discuss.elastic.co/t/curl-xget-not-returning-all-the-documents/95637/7 "2017-08-31T20:41:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.