TSVB Top hit does not show latest metric, inconsistent behaviour

Elastic Stack Kibana
1

We have a TSVB metric using the top hit aggregation. The issue we are seeing is that the top hit is not always shown by the visualisation. If a new document that would match as a top hit enters Elasticsearch and we refresh the visualisation, the metric does not change. It will change eventually, after a few minutes. We are using Kibana 6.3.0. I have not seen any release notes that mentions a fix for this particular issue.

The index pattern we use is non default (not logstash-*) and does not use @timestamp as a the timestamp field. We have tried overriding the index pattern and specifying the timestamp field. We have also tried playing around with the interval, setting it to 1m seems to make it a bit more responsive to changes but it still does not work correctly. The old metric visualisation works perfectly.

Any ideas?

2

do you have the drop last bucket setting on or off ? (under the options tab)

3

We have tried both.

I have also verified both the client and the server uses NTP and the time is synced correctly.

4

@simianhacker do you have any idea what could be the reason ?

5

If time allows the coming days, we will cross check what queries are executed by the visualisation against the Kibana API using Chrome developer tools. We will also correlate that with the query Kibana executes against Elasticsearch. I will get back to you with more information once we have done that.

6

Top Hit is always using the last bucket of a date histogram where the Kibana metric visualization is just doing a metric bucket aggregation for the entire time period. You probably want is to see either the values for the entire time range of just the last minute of data. The current implementation does an OK job at trying to show you the last value but the method isn't quite correct, it's just trying to show the last bucket, which is sometimes incomplete based on how ES calculates the starting bucket offset. Sometimes that last bucket will only contain a few seconds of data or sometimes it shows everything. The fix is to calculate the offset so the last bucket is always the last minute (or what ever you set the interval to).

https://github.com/elastic/kibana/pull/15760 fixes most of the issues we've seen with the top hits visualization in TSVB. I'm planning on working to get that merged for 6.6.

7

You might also want to set the interval to >=1m that way when you zoom out with the time picker it will switch to auto mode but when you zoom it will never make the buckets smaller the 1m.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.