Kibana data table with the aggregations only on the latest records

[Manjula_Piyumal]

Hi All,

My application is reporting one of the database table status to elasticsearch via logstash in every hour. Example structure of a document on elasticsearch side is as below,

{
"provider": "Provider 1",
"subject": "Science",
"resultsRegmonth": "2018-12",
"countForFirstPart": 153,
"countForSecondPart": 198,
"countForThirdPart": 198,
"currentLogstashIterationId": 201901020345
"@timestamp": "2019-01-02T07:45:00.593Z"
}

One provider can have multiple values for subjects and resultsRegmonth fielts. And the countFor* fields have the whole summation for the given field. So I the valid records are the one with the largest value for the currentLogstashIterationId. I want to visulize this on a kibana data table like below,

Content Provider| Sum of Count for 1st section| Sum of Count for 2nd section| Sum of Count for 3rd
Provider 1 | 201 | 148 | 150
Provider 2 | 198 | 150 | 157
Provider 3 | 180 | 144 | 150
Provider 4 | 170 | 143 | 147

So my problem here is, how to get the latest records for a given set of fields. Here I should get the latest value for each tuple of [provider, subject and resultsRegmonth] and get the sum of the countFor* fields.

Sorry if it's not very clear and thanks in advance.

Manjula

[Manjula_Piyumal]

Hi All,

Any help on this?

Thanks

[Marius_Dragomir]

The way to get the latest value of a given field is to build something like this:

Metric Aggregation: Top Hit
Field: your_field
Aggregate with: concatenate
Size: 1
Sort on: @timestamp
Order: Descending

For the other data, you need to build a dashboard that show the sum. This is how I would do it.