Multiple Group By in Time Series

Elastic Stack Kibana
1

Hello all,
I have something like a pub/sub system where an entry is added to Elasticsearch every time a subscriber is subscribed to a topic. The SubscriberAdded entry would look something like:

{
TopicId: "room1",
SubscriberId: "1"
}

So if 5 subscribers subscribed to the same topic, I would have 5 documents with the same TopicId and each with a different SubscriberId.

The goal with Kibana would be to display a time series where I can see over time the proportion of number of subscribers per topic. So at one point in time I can know for example that I have X topics with 2 subscribers, Y topics with 3 subscribers, Z topics with 3 subscribers and so on...

This seems to me that I need to:

  1. "group by" the documents by the TopicId to retrieve the count of SubscriberAdded entries for each topic.
  2. then, "group by" the count I found in #1 so that what is displayed is the different count of subscribers instead of the different topics
  3. then, do a cumulative sum of #2

From what I can see in the Time Series of Kibana, you can pipeline the aggregations but you cannot do the same with the "Group by". Is there another way to accomplish that?

2

This part of the message makes it not really clear what you expect to see in your visualization. There's a time series line chart, and each plotted point is itself a bar chart?

To show additional dimensionality in a plot, besides just X-Value and Y-Value, a "bubble chart" is pretty common visualization. The X/Y points in the chart would be a circle, where the size of the circle represents the average number of subscribers per topic and the color of the circle represents the number of topics. Or vice-versa, or whatever metrics you're really interested in.

I would recommend trying out Canvas for this plot. The expression language is really good at giving this kind of customizable control over point series data. Getting the data into Canvas might be tricky: I would probably start with looking at Canvas' Timelion data source.

3

Thanks Tim,
I will look into Canvas. I looked at Vega and it seems to be a good option as well.

4

I hope that works out for you.

There is another forum user I've been helping out, and their requirements are nearly similar to yours. Although, they are not looking for a way to view the counts as a time-based histogram: Histogram (bars) with counts as bins. The context there may help give you some ideas.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.