I have these records for 1961-2016 period. I have plotted the data corresponding to the 7th month for the entire period, together with the average aggregation for the same period.
Query:
.es(q="month:7", index=1961-2016*, timefield="@timestamp", metric='avg:T').label("Values each year from 1961 to 2016"), .es(q="month:7", index=1961-2016*, timefield="@timestamp", metric='avg:T').aggregate('avg').label("Average 1961-2016")
But what I would actually want to do is to compare the evolution through the entire period with the average of a smaller timerange - 1961 TO 1990
This is what I tried (query): .es(q="month:7", index=1961-2016*, timefield="@timestamp", metric='avg:T').label("Values each year from 1961 to 2016"), .es(q="month:7 AND year:[1961 TO 1990]", index=1961-2016*, timefield="@timestamp", metric='avg:T').aggregate('avg').label("Average 1961-1990")
And this is what I'm obtaining (plot):
Changing the query from q="month:7 AND year:[1961 TO 1990]" to q="month:7 AND year:[1961 TO 2016]" displays correct. So the problem would be that the average aggregation is calculate by dividing it to the number of years in the whole timespan, not to the number of years from the query I added.
Is there any way of accomplishing what I want? Thank you,
I changed the timelion:es.timefield Kibana advanced setting (Management -> Advanced Settings) with the string insert_date and then to other timestamp field I have - DATM, but did not manage to get anything.
I tried this query: .es(q="month:7 AND DATM:[2016/10/01 TO 2016/12/31]", index=1961-2016*, timefield="DATM", metric='avg:T').label("Values each year from 1961 to 2016"), .es(q="month:7 AND DATM:[1961/01/01 TO 1990/12/31]", index=1961-2016*, timefield="DATM", metric='avg:T').aggregate('avg').label("Average 1961-1990")
Unfortunately I don't think this is possible currently. In theory you could null out 0s if .if(eq,0,null), though that would depends on you knowing for sure that there are no 0 years, and rather a 0 represents an excluded year.
The crux of the issue is that the aggregate() function is treating null as 0. You'd need to write a new function that doesn't do that.
You could also consider filing a bug, it seems like aggregate shouldn't treat null as 0
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.