# Aggregation retuns null buckets
**URL:** https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520
**Category:** Elasticsearch
**Created:** [June 10, 2020, 1:45pm UTC](https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520 "2020-06-10T13:45:17Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)
#### Post date: [June 10, 2020, 1:45pm UTC](https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520/1 "2020-06-10T13:45:18Z")
I have one query which does aggregation on day interval for specified date range (like last month). Data is sorted based on 'MyUsage' field to return top 5 'FldNameWise'.
Problem is it retuns null aggregations for some of the day interval although data is present. If I select only specific 'FldNameWise' data then that perticular day interval have values.
Index template and other things are correct only. Not sure why null values coming in aggregation.
Kindly
Sample query:
```auto
{
"aggs": {
"FldNameWise": {
"terms": {
"field": "FldName",
"order": {
"FldOrder": "desc"
},
"size": 5
},
"aggs": {
"FldOrder": {
"avg": {
"field": "MyUsage"
}
},
"DateTimeWise": {
"date_histogram": {
"field": "MyDateTime",
"interval": "1d",
"time_zone": "abc"
},
"aggs": {
"MyUitl": {
"avg": {
"field": "MyUsage"
}
},
"MyUitl2": {
"avg": {
"field": "MyUsage2"
}
}
}
}
}
}
},
"size": 0,
"docvalue_fields": [
{
"field": "MyDateTime",
"format": "date_time"
}
],
"query": {
"bool": {
"must": [
{
"range": {
"MyDateTime": {
"gte": 123,
"lte": 456,
"format": "epoch_millis"
}
}
}
]
}
}
}
```
I am getting null values for 'MyUitl' aggregation in some day bucket even though data is present.
---
### Author:  [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)
#### Post date: [June 11, 2020, 8:43am UTC](https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520/2 "2020-06-11T08:43:32Z")
Hi, there anyone, plz help
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [June 11, 2020, 10:22am UTC](https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520/3 "2020-06-11T10:22:21Z")
> [@Harikrishna\_Sarvaiya](#):
>
> `FldName`
If you have multiple nodes/shards then some of them may not have contributed results for certain "FldName" terms to the final analysis because they weren't seen as competitive locally (a low usage average).
This [wizard](https://plnkr.co/edit/eZr7r3KZW02AxNKAHCQa?p=preview&preview) walks you through the conditions under which this may happen, how any inaccuracies might be shown and what alternative approaches to try:
In this case it may prove to be more efficient to break your search into 2 requests - one to identify the names with the most usage and then another follow-up query to fill in their low-level details.
Trying to do it all in one request may be inefficient because the `shard_size` may need increasing to retrieve more terms for fully accurate usage stats and if `shard_size` is much greater than `size` a large number of results (and all their detailed date histograms) will be thrown away.
---
### Author:  [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)
#### Post date: [June 11, 2020, 10:40am UTC](https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520/4 "2020-06-11T10:40:03Z")
Though I will explore on breaking search requests into 2 request as suggested by you.
We have only single node and single shard.
Moreover, If I select only particular value in 'FldName' or remove ` "order": { "FldOrder": "desc" },` then I get data for all day intervals.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [June 11, 2020, 10:45am UTC](https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520/5 "2020-06-11T10:45:30Z")
> [@Harikrishna\_Sarvaiya](#):
>
> We have only single node and single shard.
That should be OK then. Do you get a non-zero number for the doc\_count\_error\_upper\_bound?
What version of elasticsearch are you on?
---
### Author:  [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)
#### Post date: [June 11, 2020, 10:50am UTC](https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520/6 "2020-06-11T10:50:20Z")
Yes, I get `"doc_count_error_upper_bound": -1` , Using elasticsearch 6.7.2 version
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [June 11, 2020, 11:03am UTC](https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520/7 "2020-06-11T11:03:00Z")
Out of interest, how many unique "FldName" values do you have? (The `cardinality` agg can help reveal this).
Also, how many total docs in the index and in the time range you query on?
---
### Author:  [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)
#### Post date: [June 11, 2020, 11:10am UTC](https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520/8 "2020-06-11T11:10:52Z")
I have around only **100 unique "FldName" values**. I query on **max 30 days datetime range.**
that comprises around 30 indices as I have daily index. **One index contains around 38400** documents, so for all **30 indices it might go upto 1152000 approax.**
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [June 11, 2020, 12:09pm UTC](https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520/9 "2020-06-11T12:09:34Z")
> [@Harikrishna\_Sarvaiya](#):
>
> that comprises around 30 indices
Ah - that explains my confusion - multiple indices means multiple shards which means the potential for this distributed counting problem (even though your content is not currently physically distributed across multiple nodes).
The reported error bound of -1 means the margin for error [cannot be determined](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#search-aggregations-bucket-terms-aggregation-approximate-counts)
> "When the aggregation is either sorted by a sub aggregation or in order of ascending document count, the error in the document counts cannot be determined and is given a value of -1 to indicate this."
All of my original advice still stands.
> [@Harikrishna\_Sarvaiya](#):
>
> only **100 unique "FldName" values**.
That's not a huge number so you could try set `shard_size` to 100 to gather all the stats - your final result is only 5 so we'd throw away 95 date histograms which is less than ideal. It would be more efficient to do 2 requests but you might get away with 1 in terms of memory depending on the length of the date range and the size of date interval.
---
### Author:  [@Harikrishna\_Sarvaiya](https://discuss.elastic.co/u/Harikrishna_Sarvaiya)
#### Post date: [June 11, 2020, 1:32pm UTC](https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520/10 "2020-06-11T13:32:52Z")
Thank you so much. setting `shard_size` to 100 works. But I will explore other approach which is more efficient as you said.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 9, 2020, 1:32pm UTC](https://discuss.elastic.co/t/aggregation-retuns-null-buckets/236520/11 "2020-07-09T13:32:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.