Kibana data table with the aggregations only on the latest records

Hi All,

My application is reporting one of the database table status to elasticsearch via logstash in every hour. Example structure of a document on elasticsearch side is as below,

"provider": "Provider 1",
"subject": "Science",
"resultsRegmonth": "2018-12",
"countForFirstPart": 153,
"countForSecondPart": 198,
"countForThirdPart": 198,
"currentLogstashIterationId": 201901020345
"@timestamp": "2019-01-02T07:45:00.593Z"

One provider can have multiple values for subjects and resultsRegmonth fielts. And the countFor* fields have the whole summation for the given field. So I the valid records are the one with the largest value for the currentLogstashIterationId. I want to visulize this on a kibana data table like below,

Content Provider| Sum of Count for 1st section| Sum of Count for 2nd section| Sum of Count for 3rd
Provider 1 | 201 | 148 | 150
Provider 2 | 198 | 150 | 157
Provider 3 | 180 | 144 | 150
Provider 4 | 170 | 143 | 147

So my problem here is, how to get the latest records for a given set of fields. Here I should get the latest value for each tuple of [provider, subject and resultsRegmonth] and get the sum of the countFor* fields.

Sorry if it's not very clear and thanks in advance.


Hi All,

Any help on this?


The way to get the latest value of a given field is to build something like this:

Metric Aggregation: Top Hit
Field: your_field
Aggregate with: concatenate
Size: 1
Sort on: @timestamp
Order: Descending

For the other data, you need to build a dashboard that show the sum. This is how I would do it.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.