Graphing Collectd Metrics in Kibana 4

Elastic Stack Kibana
1

Hi,

Have collectd sending logs to Logstash but cant get the graphs for the metric value.

This is my search:
type: "collectd" AND plugin: "memory" AND type_instance: "used" AND value: ""

Is there a better way to search or create the graph ?

Thanks
Aidan

2

Hi Aidan,

Maybe a bit more information would help me. What are the steps you're going through in Kibana?

Thanks,
Lee

3

Hi Lee,

First step is using Discovery menu to create a search but I don't have the correct syntax for getting the memory value ?

This is the search I'm trying to use:
type: "collectd" AND plugin: "memory" AND type_instance: "used" AND value: ??

If I try to use Dashboards it wants to use either a search or visualization which I don't have at the moment ?

Thanks
Aidan

4

Am I missing a plugin for Kibana to run the collectd graphs ?

5

Hi Aidan,
No, I don't think there's any plugin for collectd graphs.

I think you're right to stick to the Discover tab until you find you're getting the metrics you're looking for. Then go to Visualize to pick what kind of charts you want, and finally Dashboard to put those visualizations together.

When you go to Discover, I would enter each of your search criteria one at a time and watch the count of hits in the top right corner of the chart. If you add a search criteria and your hits go to 0 you know there's a problem.

And I think you might be doing something wrong on your last "value:??". The search is used to filter out the documents you want (not the data you want from those documents). Try leaving that off.
Try instead to find the "value" field in the left pane on the discover tab and click [Add] on that. That should show you the data under the bar graph.

Save that search and then go to Visualize and select from that saved search.

Maybe a line chart would be good? On the Y-Axis change it from Count to Sum and then the Field selection list appears below it. Here you might pick "value" if that field has the memory used value.

The in the buckets pick X-Axis,
Aggregation: Date Histogram
Field: @timestamp (if you have that)
Interval: Auto
and click the green "Go" arrow

Does that show you anything like what you're looking for?

Lee

6

Many thanks Lee, this worked just as you explained...

I just have one more issue, how to display the values as percentages particularly for cpu and memory ?

I have also noticed that the cpu value seem to be continuously increasing , can't see to figure it out. I'm using collectd version-5.5 installed from tar file.

This is my cpu aggregates plugin:

LoadPlugin aggregation

<Plugin "aggregation">

Plugin "cpu"
Type "cpu"
GroupBy "Host"
GroupBy "TypeInstance"
CalculateAverage true

LoadPlugin "match_regex"

<Chain "PostCache">


Plugin "^cpu$"
PluginInstance "^[0-9]+$"


Plugin "aggregation"

Target stop

Target "write"

Thanks
Aidan

7

Hi Aidan,
I'm not sure what the issue is with the increasing values. Maybe you could paste 1 or 2 documents here so we could see the cpu data in them? I'll take a look at Collectd and see if I can easily set up a test.

Lee

8

Hi Aidan,
I also found this post https://mtalavera.wordpress.com/2015/02/16/monitoring-with-collectd-and-kibana/.
It looks like it was done with older versions of Elasticsearch and Kibana but it might still give you the clues you need.

Lee

9

I am using RedHat 7 + kibana 5 + ES 5 + LS 5 and collectd and cannot get it to work.
I am trying to graph a file system usage:

host:syssatpl1 && type_instance:var

the selected field is - value

the search works but virtualization does not. By the way, is there a way to set the values of to Y axis or this is done automatically?

Thanks
MarkD

10

@markd Please start a new thread for your question.