How can I make collectd report which device/mount point it is looking at?

Elastic Stack Logstash
1

Using collectd i want to report disk space usage to ELK. I successfully enabled CollectD's df plugin, and it is sending data to ELK. But I don't see any field that says "this is the usage for /boot", or "this is the usage for /home", or any other mountpoint I tell collectd to monitor.

I collectd only able to report total disk space usage? I was trying something like:

<Plugin "df">
  MountPoint "/boot"
  MountPoint "/"
</Plugin>
2

It should automatically do that. The resulting fields in Logstash will be "plugin_instance" (which will show the mount point), "type_instance" (e.g. free, used, reserved), and the associated "value."

For example, my Kibana query:

type.raw:"collectd" AND plugin.raw:"df" AND plugin_instance.raw:"root"

And an individual "log line" from the Discover panel:

plugin:df plugin_instance:root type:collectd tags:collectd host:blackbox.REDACTED.net @timestamp:October 21st 2015, 17:37:48.159 type_instance:free collectd_type:df_complex value:846,273,490,944 @version:1 _id:AVCMxAX_4IhnevJIINXz _type:collectd _index:logstash-2015.10.21
1 Like
3

And now I see that field... I swear I didn't see it before...

sigh

Thanks. :smile:

4

Hmm... I'm having issues graphing the data in a useful manner. df is giving me several values for type_instance. root, boot, reserved, used, and free.

If it's root or boot, I get two fields called used and free.

If it's reserved, used, or free, I get one field called value.

And, after some research while writing this post, I figured out that the difference is because of different collectd versions. Ubuntu 12.04 has 4.10.1, and Ubuntu 14.04 has 5.4.0.

CollectD 4.10.1 has the data with the used and free fields.

"_source": {
    "host": "hostname",
    "@timestamp": "2015-10-23T23:05:14.000Z",
    "type_instance": "root",
    "plugin": "df",
    "collectd_type": "df",
    "used": 5152854016,
    "free": 14655479808,
    "@version": "1",
    "type": "collectd"
  },

5.4.0 has the data with the value field.

"_source": {
    "host": "hostname2",
    "@timestamp": "2015-10-23T23:05:08.225Z",
    "type_instance": "free",
    "plugin": "df",
    "plugin_instance": "boot",
    "collectd_type": "df_complex",
    "value": 58056704,
    "@version": "1",
    "type": "collectd"
},

So how do I build a decent visualization? I'd like a split line graph that shows the max values for free and used. That's easy to do with data from 4.10. But how do I add in the data from 5.4? It's a separate doc for each free or used value... I could do a version specific visualization for 5.4, but that's rather clunky. Is there a way to use the scripting feature to do this?

Any thoughts?

5

If I share this:

https://kibana.host.tld/#/visualize/edit/Used-and-Free-Disk-Space-Split-Lines?_g=%28%29&_a=%28filters:!%28%29,linked:!f,query:%28query_string:%28analyze_wildcard:!t,query:%27type.raw:%22collectd%22%20AND%20plugin.raw:%22df%22%20AND%20%28plugin_instance.raw:%22root%22%20OR%20type_instance.raw:%22root%22%29%27%29%29,vis:%28aggs:!%28%28id:%271%27,params:%28field:value%29,schema:metric,type:max%29,%28id:%272%27,params:%28customInterval:%272h%27,extended_bounds:%28%29,field:%27@timestamp%27,interval:auto,min_doc_count:1%29,schema:segment,type:date_histogram%29,%28id:%273%27,params:%28filters:!%28%28input:%28query:%28query_string:%28analyze_wildcard:!t,query:%27type_instance.raw:%22free%22%20OR%20plugin_instance.raw:%22free%22%27%29%29%29,label:%27%27%29,%28input:%28query:%28query_string:%28analyze_wildcard:!t,query:%27type_instance.raw:%22used%22%20OR%20plugin_instance.raw:%22used%22%27%29%29%29%29%29%29,schema:group,type:filters%29,%28id:%274%27,params:%28field:used%29,schema:metric,type:max%29,%28id:%275%27,params:%28field:free%29,schema:metric,type:max%29%29,listeners:%28%29,params:%28addLegend:!t,addTimeMarker:!f,addTooltip:!t,defaultYExtents:!f,drawLinesBetweenPoints:!t,interpolate:linear,radiusRatio:9,scale:linear,setYExtents:!f,shareYAxis:!t,showCircles:!t,smoothLines:!f,times:!%28%29,yAxis:%28%29%29,type:line%29%29

Can you figure out the Kibana 4 settings I've tried? Since taking a screenshot doesn't capture all of the settings on the Visualization tab.

6

collectd has a "target_v5upgrade" plugin which helps converting v4 metrics to v5 format. See https://collectd.org/wiki/index.php/V4_to_v5_migration_guide.

Other than that, the collectd version shipped with 12.04 is really old and riddled with bugs. My advice would be to upgrade to the latest release: https://launchpad.net/~collectd/+archive/ubuntu/collectd-5.5

Sure ES/kibana can do a lot of things, but everything is much easier when the input data is nice and consistent :wink:

My 2¢

7

Agreed. Hadn't looked for a good ppa yet. Thanks.

Know of a good repo for CentOS 6? I haven't found one yet.

8

Unfortunately, no. I get a Could not locate that visualization (id: Used-and-Free-Disk-Space-Split-Lines) error.

9

I start with this:

Visualize tab:
Query bar: type.raw:"collectd" AND plugin.raw:"df" AND plugin_instance.raw:"root"

Screenshot 2015-10-26 13.51.41.png350×640 27.3 KB

I get a pie chart here with the slices being the amounts used by the root device.

10

Hi Jerrac,

i am also facing this issue.

I am able to get data in logstash logs and on stdout but as mentioned by @theuntergeek, new fields are not visible on GraylogUI.