Kibana 4 transfer rate line chart

Elastic Stack Kibana
1

Hi everyone, not sure if this is the correct place to ask questions - but here goes.

I have data in elastic search relating to network data. Netflow data if you will.
Ip1 ->number of bytes -> ip2

I'd like to graph the transfer rate in a line chart , but cant figure out how to achive this. Maybe its not possible (yet?)

Obviously I can add a scripted field (or just precalculate when indexing the document) - bytes * 8 / interval = bits per second.

But because the x-axis interval changes - this calculation doesnt work if the graph interval does not align with my data/scripted field interval.

The second part to is that Ideally I'd like the y-axis to display in b/s , kb/s, mb/s etc.
Scripted fields only seem to support Bytes.

Regards.
Pieter

2

If I'm understanding you correctly, your documents look something like this:

{
  "from": "192.168.0.1",
  "to": "192.168.0.2",
  "start": 1431623198
  "end": 1431623205
  "size": 123503
}

Are you currently able to do get the data you want graphed with elasticsearch aggregations?

If you were indexing samples of the current bytes per second we could show you the average over time, but I don't think that Elasticsearch can calculate the raw metrics necessary for this type of chart given your current data.

3

Yes, your assumptions about my data is correct... but I am creating and indexing the data myself, so I CAN add calculated fields. But my problem is that the calculations should be made based on the date histogram interval (which I dont want to make fixed for obvious reasons)

No, I do not know how to go about getting the desired results with an elastic search aggregation as (as above) the calculation needs to be based on the interval granularity.

Using an average of precalculated bits/s values would only work (I think) if there are entries for every interval.
I.e. If 192.168.0.1 sent at 10Mb/s to 192.168.0.2 , but only for a 5 minute period , then the data in 5min or less granularity , the average should work - but viewing at larger intervals would still show the two ips sending at 10Mb/s for an hour for instance.