Hi All,
My application is reporting one of the database table status to elasticsearch via logstash in every hour. Example structure of a document on elasticsearch side is as below,
{
"provider": "Provider 1",
"subject": "Science",
"resultsRegmonth": "2018-12",
"countForFirstPart": 153,
"countForSecondPart": 198,
"countForThirdPart": 198,
"currentLogstashIterationId": 201901020345
"@timestamp": "2019-01-02T07:45:00.593Z"
}
One provider can have multiple values for subjects and resultsRegmonth fielts. And the countFor* fields have the whole summation for the given field. So I the valid records are the one with the largest value for the currentLogstashIterationId. I want to visulize this on a kibana data table like below,
Content Provider| Sum of Count for 1st section| Sum of Count for 2nd section| Sum of Count for 3rd
Provider 1 | 201 | 148 | 150
Provider 2 | 198 | 150 | 157
Provider 3 | 180 | 144 | 150
Provider 4 | 170 | 143 | 147
So my problem here is, how to get the latest records for a given set of fields. Here I should get the latest value for each tuple of [provider, subject and resultsRegmonth] and get the sum of the countFor* fields.
Sorry if it's not very clear and thanks in advance.
Manjula