# Lucene vs Elastic Search Document Count difference and its impact on term aggregation buckets
**URL:** https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827
**Category:** Elasticsearch
**Created:** [July 19, 2023, 8:57pm UTC](https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827 "2023-07-19T20:57:12Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@S\_Star](https://discuss.elastic.co/u/S_Star)
#### Post date: [July 19, 2023, 8:57pm UTC](https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827/1 "2023-07-19T20:57:12Z")
We have an index with lot of auto generated data for load testing and we noticed that there is significant difference in Elasticsearch doc count (using the \_count API) vs /indices API e.g
ES documents : ~80 million
Lucene documents : ~2 billion
There is only single nested field in the document so can someone explain what might be causing such a huge inflation in Lucene document count.
Also If I run an term aggregation on this , does this impacts the overall number of buckets which gets created in memory.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 20, 2023, 4:32am UTC](https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827/2 "2023-07-20T04:32:27Z")
How many object does your nested field hold on average? Each nested object is stored as a separate document behind the scenes and does show up in the indices stats.
---
### Author:  [@S\_Star](https://discuss.elastic.co/u/S_Star)
#### Post date: [July 20, 2023, 10:59am UTC](https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827/3 "2023-07-20T10:59:48Z")
On an average or 2, 1 hence not sure what might be the cause
---
### Author:  [@S\_Star](https://discuss.elastic.co/u/S_Star)
#### Post date: [July 21, 2023, 11:37am UTC](https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827/4 "2023-07-21T11:37:55Z")
@Christian_Dahlqvist Any further insight into this ?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 21, 2023, 1:00pm UTC](https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827/5 "2023-07-21T13:00:44Z")
It might help if you show the exact output from the APIs. Do you have a lot of updated and/or deleted documents that show up in the indices api but not in count?
---
### Author:  [@S\_Star](https://discuss.elastic.co/u/S_Star)
#### Post date: [July 21, 2023, 1:40pm UTC](https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827/6 "2023-07-21T13:40:33Z")
In this case no updated and/or deleted document count
Indices API output :
green, open, index name, uuid, 5, 1, 2450791167, 0, 239.6gb, 119.8gb
Count API output :
> Blockquote
> {
> "count" : 88336911,
> "\_shards" : {
> "total" : 5,
> "successful" : 5,
> "skipped" : 0,
> "failed" : 0
> }
> }
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 21, 2023, 5:58pm UTC](https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827/7 "2023-07-21T17:58:54Z")
My guess is that you have more nested documents than you think you do.
---
### Author:  [@S\_Star](https://discuss.elastic.co/u/S_Star)
#### Post date: [July 21, 2023, 8:50pm UTC](https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827/8 "2023-07-21T20:50:17Z")
Thanks for getting back @Christian_Dahlqvist , Does these nested documents affect term aggregations even when i am not explicitly querying for these nested fields ?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 22, 2023, 5:36am UTC](https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827/9 "2023-07-22T05:36:25Z")
No, not that i know of.
It may be worthwhile creating a runtime field containing the size of the nested array or maybe even index it as a proper field using an index pipeline. That way you would be able to aggrgate and get statistics on it and see if it all adds up.
---
### Author:  [@S\_Star](https://discuss.elastic.co/u/S_Star)
#### Post date: [July 23, 2023, 8:10pm UTC](https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827/10 "2023-07-23T20:10:39Z")
thanks for the update
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 20, 2023, 8:10pm UTC](https://discuss.elastic.co/t/lucene-vs-elastic-search-document-count-difference-and-its-impact-on-term-aggregation-buckets/338827/11 "2023-08-20T20:10:43Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.