Strange aggregation behavior (returning partial data)

Hello there

Elastic aggregation returns sometimes partial data, sometimes full data from the same documents depending on date range.
Ex. docs:

"message": {
      "metadata": {
        "kitId": "07e102c2-c92c-4a9a-9d95-a7f89d006edd",
        "contentId": "1328"
      },
      "type": "APP_METRICS",
      "elasticsearch": {
        "index": "2019.03.28",
        "timestamp": "2019-03-28T14:13:49.177Z"
      },
      "payload": {
        "metric": {
          "name": "What do you expect to see here (test)?",
          "id": "b4ff6780-5163-11e9-9663-5d217198464c",
          "value": 1,
          "subname": "Fitness User"
        }
      }
    }

Multiple logs with fields "name", "subname", "value"
All of them generated with small time difference (ex. all from 28 march 2-3pm)

Aggregations are being made using mseach endpoing. Something like this:

"aggs": {
    "metrics": {
      "terms": {
        "field": "message.payload.metric.name.keyword",
        "size": 10000
      },
      "aggs": {
        "submetrics": {
          "terms": {
            "field": "message.payload.metric.subname.keyword"
          }
        },
        "data": {
          "stats": {
            "field": "message.payload.metric.value"
          }
        }
      }
    }
  }

And if I take timestamp range for example from 1553731200000 (0.00 28 march) to 1553806740000 - I will get result with sub-aggregation:

[
    {
        "name": "What do you expect to see here (test)?",
        "data": {
            "count": 8,
            "min": 1,
            "max": 1,
            "avg": 1,
            "sum": 8
        },
        "submetrics": [
            {
                "name": "Apparel Store!",
                "count": 5
            },
            {
                "name": "Children’s Store",
                "count": 1
            },
            {
                "name": "Cosmetics Store",
                "count": 1
            },
            {
                "name": "Fitness User",
                "count": 1
            }
        ]
    }
]

But if I extend "from" time to something less than 1553731200000 (ex. 1553731100000) - I'd expect to see at least same result. But I receive SAME(!) documents without subaggregation:

[
    {
        "name": "What do you expect to see here (test)?",
        "data": {
            "count": 8,
            "min": 1,
            "max": 1,
            "avg": 1,
            "sum": 8
        }
    }
]

Struggling already for 2 days.
Any ideas why is this happening? Thanks in advance

Could you provide a reproducible example with mappings and data? It is hard to understand what is going on otherwise.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.