Wrong data aggregation in a table

Hello,
I've created a table, but the data in the table differ from the elastic.
For example, for player_id "750" there are 6 records from 2 different devices:


However it's impossible because each player's id is uniquely connected with a device: 1 device = 1 player_id and and vice versa. In Elastic with the id = 750 only one device is connected:

The same problem I've encountered at various IDs: e.g., id 744. In a data table we see 5 different devices connected with the id:

But in Elastic there is only one device:

What it can be caused by? Recently I've updated Kibana from 7.3 up to 7.4, so can the problem be connected either with the Kibana 7.4 version or with the incorrect upgrade from my side?

Below I attach the request of the table vizualization:

{
  "aggs": {
    "2": {
      "terms": {
        "field": "device.keyword",
        "order": {
          "1": "desc"
        },
        "missing": "__missing__",
        "size": 5000
      },
      "aggs": {
        "1": {
          "avg": {
            "field": "stage_duration"
          }
        },
        "3": {
          "histogram": {
            "field": "player_id",
            "interval": 6,
            "min_doc_count": 1
          },
          "aggs": {
            "7": {
              "histogram": {
                "field": "tutorial_id",
                "interval": 5,
                "min_doc_count": 1
              },
              "aggs": {
                "6": {
                  "histogram": {
                    "field": "stage_id",
                    "interval": 1,
                    "min_doc_count": 0
                  },
                  "aggs": {
                    "4": {
                      "terms": {
                        "field": "completed",
                        "order": {
                          "1": "desc"
                        },
                        "size": 5
                      },
                      "aggs": {
                        "1": {
                          "avg": {
                            "field": "stage_duration"
                          }
                        },
                        "5": {
                          "terms": {
                            "field": "created_at",
                            "order": {
                              "1": "desc"
                            },
                            "size": 50
                          },
                          "aggs": {
                            "1": {
                              "avg": {
                                "field": "stage_duration"
                              }
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "size": 0,
  "_source": {
    "excludes": []
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {},
  "docvalue_fields": [
    {
      "field": "created_at",
      "format": "date_time"
    },
    {
      "field": "player_created_at",
      "format": "date_time"
    }
  ],
  "query": {
    "bool": {
      "must": [],
      "filter": [
        {
          "bool": {
            "filter": [
              {
                "bool": {
                  "should": [
                    {
                      "match": {
                        "is_tester": false
                      }
                    }
                  ],
                  "minimum_should_match": 1
                }
              },
              {
                "bool": {
                  "should": [
                    {
                      "match": {
                        "build": "0.3.0.1650"
                      }
                    }
                  ],
                  "minimum_should_match": 1
                }
              }
            ]
          }
        },
        {
          "range": {
            "stage_id": {
              "gte": 16,
              "lt": 19
            }
          }
        },
        {
          "range": {
            "created_at": {
              "format": "strict_date_optional_time",
              "gte": "2019-12-10T22:00:00.000Z",
              "lte": "2019-12-13T21:30:00.000Z"
            }
          }
        }
      ],
      "should": [],
      "must_not": []
    }
  }
}

If you want to build a table using individual documents without aggregation, you should save the table you see in Discover and then embed that in a dashboard.

I've built a table using data from elactic, as usual. And I see that the result doesnt correspond the reality.
What do you mean by " build a table using individual documents"?

The table you showed in the first screenshot is an aggregated table- it's not representing individual documents, so if you expect 1:1 correlation between the rows in the table and documents in Elasticsearch you can only do that with Discover

So can you explain please why the same request returns different results in aggregated table and in Discover? (data frame is the same)

It's not the same request at all- try running the request in the Kibana dev tools and you'll see that it shows aggregated data, not individual documents

The solution was to change the "player_id" bucket type from Histogram to Terms

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.