Moving average values differ in elasticsearch and Kibana

[Sureya (Sureya Sathiamoorthi)]

I am using Kibana to visualise a POC on monitoring total transactions occurred and compares it with moving average of window 10, to raise alerts.

My alerting code is in Python which queries elastic search (using index name) and redirects users to Kibana to visualise the change in trend. I see that Moving average values calculated in Kibana differs from the moving average values I from elastic search queries. As a consequence, our alerting system alerts for an anomaly but visualisation isn't plotting it as errors.

Can someone please explain how I can sync both moving average values !?

[simianhacker (Chris Cowan)]

What are you using to visualize the moving average? Timelion?

[simianhacker (Chris Cowan)]

The other thought I just had come to mind is what is the interval you're using in Kibana vs. Elasticsearch? Window will be the number of buckets the interval creates. So a window of 10 on a 1 hour chart with 10 second interval will give you a different result then a window of 10 on a 24 hour chart with a 1 hour interval.

[Sureya (Sureya Sathiamoorthi)]

Sorry for the confusion. Yes the Visualization is in Timelion in Kibana

In Both Timelion and in Elastic search queries the data is aggregated over 2 mins , data considered is past 24 hrs of data

Timelion formula

.es(index="local-live_data",timefield="created_at",metric="count:created_at",q="event_type=event.started_transaction").color("green"),.es(index="local-live_data",timefield="created_at",metric="count:created_at",q="event_type=event.started_transaction").movingaverage(10).sum(15).color("red"),.es(index="local-live_data",timefield="created_at",metric="count:created_at",q="event_type=event.started_transaction").movingaverage(10).subtract(15).color("red")

ES query

GET local-live_data/_search
{
"size": 0,
"query":{
"bool":{
"must":[
{
"range":{
"created_at":{
"gte":"now-24h",
"lte": "now"
}
}
},
{
"match":{
"event_type":"event.started_transaction"
}
}
]
}
},
"aggs":{
"my_date_histo":{
"date_histogram":{
"field":"created_at",
"interval":"2m",
"time_zone": "Europe/London"
},
"aggs":{
"the_count":{
"value_count": {
"field": "created_at"
}
},
"the_movavg":{
"moving_avg":{
"buckets_path":"the_count",
"window":10
}
},
"final_filter":{
"bucket_selector":{
"buckets_path":{
"TheCount":"the_count",
"TheMovAvg":"the_movavg"
},
"script": "params.TheCount > (params.TheMovAvg == null ? 0 : params.TheMovAvg)+15 || params.TheCount < (params.TheMovAvg == null ? 0 : params.TheMovAvg)-15 "
}
}
}
}
}
}