Our business problem is to find the duration of a session, and our sample simplified data set is a simple table with 2 columns - request_date and session_id. All we need is to find the max(request_date), min(request_date) and subtract them.
We create Rollup Jobs and in Terms, we choose session_id and Metric, we choose min and max of request_date and we try querying it in Dev Tools
The actual values of min, max (in epoch) are 1565744459000,1565744505000
and values of min, max returned from rollup are 1565744431104,1565744562176.
So a 46.000 sec duration, became 131.072 secs.
Can someone explain why this really weird discrepancy occurs while using Rollup Jobs?
There's also a difference in what _source returns and what docvalue_fields returns(as seen the picture below in request.date.min.value, request.date.max.value)
{
"_index" : "1234_rollup",
"_type" : "_doc",
"_id" : "1234_rollup$oLCd6lhqSwUI_dCTGrgYgA",
"_score" : null,
"_source" : {
"request_date.max.value" : 1.565744505E12,
"request_date.min.value" : 1.565744459E12,
"_rollup.version" : 2,
"request_date.date_histogram._count" : 4,
"request_date.date_histogram.time_zone" : "UTC",
"session_id.terms._count" : 4,
"request_date.date_histogram.timestamp" : 1565740800000,
"request_date.date_histogram.interval" : "24h",
"session_id.terms.value" : "07a954ff-186f-45f9-9bc4-8b53b1613580",
"_rollup.id" : "1234_rollup"
},
"fields" : {
"request_date.max.value" : [
1.565744562176E12
],
"request_date.min.value" : [
1.565744431104E12
]
}