Kibana Aggregation Intervals in Data Table (Pivot?)


(Michael Hooper) #1

I've got a bunch of network traffic data that looks like this

src | dest | duration (seconds)
----|------|------------------
  a |   b  | 1.37 
  a |   c  | 0.44
  b |   a  | 2.32
  c |   b  | 1.73

I need to divide out this data by src and duration per .5 second interval (anything over 3 can be put in a 3+ column), I don't care about dest. So ideally I'd have a table that has src down the left hand side and duration intervals across the top and the counts of transactions per interval in the cells.

The end result I'm looking will look something like this:

src | 0 to .5 | .5 to 1 | 1 to 1.5 | 1.5 to 2 | 2 to 2.5 | 2.5 to 3 | 3+
----|---------|---------|----------|----------|----------|----------|---------
 a  |  275    |    120  |    32    |    12    |    4     |   3      |   7
 b  |  315    |    105  |    75    |    28    |    16    |   8      |   12
 c  |  198    |    113  |    71    |    102   |    173   |   113    |   87

Is this basically a pivot table? Is there a way to do that in Kibana? Any help would be appreciated.


(CJ Cenizal) #2

Hi Michael, unfortunately you can't use Kibana to generate a table like that. Here are a couple outstanding request for pivot-table-like functionality, which you can watch for progress:

Thanks,
CJ


(CJ Cenizal) #3

I talked with @LeeDr a bit about this and it seems like the closest you can get to what you're looking for is a Terms bucket aggregation on your "src" field, and then a Range sub-aggregation on the "duration" field for the different ranges you're interested in.


(Michael Hooper) #4

Yeah, unfortunately, that doesn't really get me where I want, I'll still have to pull it into excel and pivot it to get what I'm wanting. The idea is a quick glance to see if there are certain sources that are having a large amount of long transactions. I might be able to get what I want from a heat map plugin I'm looking at. I'm about to start playing around with that, because I just don't see anything that offers a pivot table equivalent in Kibana. Thanks though.


(system) #5