Network Interface Utilization Visualization

Elastic Stack Kibana
1

Hi Everyone,

I'm new to the ELK stack and have formerly used commercial monitoring products to achieve the task I'm now placed with but I'm looking towards monitoring network appliances for their network utilization using SNMP.

My setup is:
collectd - SNMP plugin polls out to the appliances. At the moment, looking at the IF-MIB HC Octet counters.
logstash - Binary input, no filters, output ES.
ES - 2.3.3

Steep learning curve, but so far a lot of this seems intuitive which is great.

Unfortunately I've come unstuck on how I might achieve the following step and spent many hours trying to find any such existing article or blog post out there which might shine some light.
So I'm looking for a push in the right direction, even if it's a link to where this may have been asked before (which I've just not found so far)

Here's what I'm seeing:

ES sees documents which have RX and TX, and they basically just keep accumulating octets as delivered by the MIB.
So Logstash and Collectd are working fine.

I need to graph/visualize the difference between the values in a previous document with the later one, and then work out how much change there was during the interval between them.

i.e.

The first document, TX = 1000
The second document, TX = 6000
Octets/Bytes per second (based on a five second interval) = 1000 Bps (8000 bps).

Would this be something achieveable with scripted fields in Kibana, or am I too far gone and need to look at ES more directly?

I suppose I'm looking to get the delta value between the two to plot a graph.

Many thanks,

jdmac

2

Hi @jdmac. If you need to graph and run computations on individual values, Timelion (a Kibana plugin) is going to be your best bet. Core Kibana is really built around running aggregations on your data set and displaying metrics for buckets of documents.

3

Hi @Bargs , thanks for your reply.
I had looked at Timelion but there appears to be little documentation on it and I couldn't get my head around how I'd proceed to get that to perform what I'm after.

Is there any official documentation you can point me towards for Timelion?

4

You're right, there's not a ton of documentation at the moment. There are a couple of videos that might be helpful:

https://www.elastic.co/videos/timelion-plugin-for-kibana-enables-times-series-paris-meetup
https://www.elastic.co/videos/time-series-analysis-using-timelion

There's also some in app documentation, if you fire up timelion it'll start by walking you through a tutorial which includes a reference of all the available functions.

Let me know if you're still having trouble after that. I'm not a timelion expert myself but I should be able to find someone who can help you out.

5

Thanks again @Bargs,

The second video, I'd already watched but the first seemed to add some value.

Problem is, the output I'm getting still doesn't seem to be what I expect.

If I use a true TSDB I get what I need, but the problem is our architects are gearing towards using ES for log aggregation and TSDB functions, so I really don't have much say in it.

I'm just trying to work out the difference between one metric and the next and division it by the time interval between.

I tried this, which seems to have come closest, but it keeps coming out with negative values intermittently for which I can't figure out (and the values are as expected, increasing all the time). TSDBs don't seem to have that issue.

.es(metric=max:tx).subtract(.es(metric=max:tx, offset=-1m))
.es(metric=max:rx).subtract(.es(metric=max:rx, offset=-1m))

Additionally tried what I believe would be more correct:

.es("host:Vyos AND type_instance:eth0", avg:rx).derivative() .es("host:Vyos AND type_instance:eth0", avg:rx).derivative()

6

Hi jdmac, i have tried to, and I cannot make it work, im getting the summatoryor zero values., How could it be done? the math seems to be quite simple.
Thanks , diego pereyra