Interaction between Multiple Date Histograms (based on different date fields)

Elastic Stack Kibana
1

We've got Twitter data in the form of tweets in ES and I'm building views for analyzing them.

Different timelines (histograms with tweet.created_at) showing tweets over time are used to spot patterns, outliers or strange things.

It's also useful to analyse user.created_at together with these timelines. For example, selecting a spike in a histogram over user.created_at to filter the tweet timelines to see what these users are tweeting.

My index pattern has tweet.created_at as the time-field. There's no problem in creating histograms based on this field.

But when I try to create a histogram based on user.created_at I get strange problems. First, the value "auto" is removed in the interval setting. And when setting the interval manually I get an empty view. A "solution" I found here was to create an alias index in ES and then a new index pattern for that one in Kibana with user.created_at as time-field. This solution allowed me to have both types of histograms in the same dashboard: tweets over time and users over time.

But the problem is that both histograms are filtered to the time filter, so if I want to see one year of tweets I also am limited to seeing users created only this year. A "solution" I found here was to set a very wide time filter. But the problem is that the two histograms are connected via two different fields (tweet.created_at and user.created_at) and the filtering therefore gets completely wrong for my purpose. If I select a certain two days worth of tweets in the tweets histogram the result in the user histogram would be users created these two days, and NOT the distribution over time of user_created_at for the selected tweets (like I want).

How do I solve this problem?

2

Hi @Albinsson,

the date histogram on the non-timefield works fine for me. Which version of the Elastic Stack are you using? It seems this is a known issue that is being discussed: https://github.com/elastic/kibana/issues/5707 If you would add a description of your problems to the issue, it might help the developers that work on it.

3

Thanks. We're using version 5.1.1. If I understand the link you sent it should work in latest version. I guess we need to try that.

4

With latest version (6.0.1) it works to create date histograms with other date fields.

Unfortunately, since you still cannot have "auto bucketing" on any other field than the time-field the usefulness is limited. Example use case: A date histogram shows the distribution of user.created_at covering 10 years. You see a thin spike and filter to have a closer look and end up with only two bars, hiding the information where the true spike is.

5

Yes, I agree that this is not ideal. Have you considered using the "Timelion" or "Visual Builder" visualization types for your charts? They are more flexible in regard to choosing the primary time field since they don't rely on index-patterns.

6

I will check them out. Thanks.

7

I've now tried Timelion.

The code for the simple visualization is:

.es(metric='cardinality:tweet.user.id',kibana=true,timefield=tweet.user.created_at).bars()

It does create the histogram I want, showing the distribution of when users were created. And I can zoom with autobucketing. However, when I add this Timelion visualization to a dashboard it will get connected to the global time filter of the dashboard which means that both tweet.created_at and tweet.user.created_at will be tied to the time filter and thereby I end up with the same problem as before with separate index patterns.

Is there a way to force the Timelion visualization to not care about the time filter and perhaps hard code a time period in the code?

8

I don't know of any way to exempt any panel from the global time filter. :frowning:

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.