Hello there
Elastic aggregation returns sometimes partial data, sometimes full data from the same documents depending on date range.
Ex. docs:
"message": {
"metadata": {
"kitId": "07e102c2-c92c-4a9a-9d95-a7f89d006edd",
"contentId": "1328"
},
"type": "APP_METRICS",
"elasticsearch": {
"index": "2019.03.28",
"timestamp": "2019-03-28T14:13:49.177Z"
},
"payload": {
"metric": {
"name": "What do you expect to see here (test)?",
"id": "b4ff6780-5163-11e9-9663-5d217198464c",
"value": 1,
"subname": "Fitness User"
}
}
}
Multiple logs with fields "name", "subname", "value"
All of them generated with small time difference (ex. all from 28 march 2-3pm)
Aggregations are being made using mseach endpoing. Something like this:
"aggs": {
"metrics": {
"terms": {
"field": "message.payload.metric.name.keyword",
"size": 10000
},
"aggs": {
"submetrics": {
"terms": {
"field": "message.payload.metric.subname.keyword"
}
},
"data": {
"stats": {
"field": "message.payload.metric.value"
}
}
}
}
}
And if I take timestamp range for example from 1553731200000 (0.00 28 march) to 1553806740000 - I will get result with sub-aggregation:
[
{
"name": "What do you expect to see here (test)?",
"data": {
"count": 8,
"min": 1,
"max": 1,
"avg": 1,
"sum": 8
},
"submetrics": [
{
"name": "Apparel Store!",
"count": 5
},
{
"name": "Children’s Store",
"count": 1
},
{
"name": "Cosmetics Store",
"count": 1
},
{
"name": "Fitness User",
"count": 1
}
]
}
]
But if I extend "from" time to something less than 1553731200000 (ex. 1553731100000) - I'd expect to see at least same result. But I receive SAME(!) documents without subaggregation:
[
{
"name": "What do you expect to see here (test)?",
"data": {
"count": 8,
"min": 1,
"max": 1,
"avg": 1,
"sum": 8
}
}
]
Struggling already for 2 days.
Any ideas why is this happening? Thanks in advance