Kibana visualization - different ways to show filesystem usage

[arun_prasath1 (Arun Prasath)]

I am suppose to show all the filesystem (local and mounted) usage in kibana. I could not understand how disk usage (free, available, total) can be shown from the files available using Kibana. I mean what visualization will work for filesystem usage, what fields to use in Y-axis and what sub-bucket type to use. Any help is appreciated.

[Marius_Dragomir]

Hello,

The best way to display this can be found inside the Metricbeat dashboards and you can see them in work easily by doing this:

1. Download and start Metricbeat (it is configured to ship logs to localhost by default, so no config would be needed in that case)
2. Install the Metricbeat dashboards. There is a script called import_dashboards inside your metricbeat/scripts folder.
3. Open the Kibana dashboard named Metricbeat filesystem per Host and you can see in the bottom of the dashboard a Data Table that shows disk space usage by drive, with total, available, used and used % stats.
   It looks like this on my Windows machine:
   
   

[arun_prasath1 (Arun Prasath)]

@Marius_Dragomir

Open the Kibana dashboard

Marius_Dragomir @ I could see the filesystem usage for local filesystes but the server name is missing. I have multiple servers pushing metricbeat logs to ELK. How to get the filesystem usage for each and every server separately with servername. Here I do not know which server it is showing the filesystem usage. Correct me if I am seeing wrongly.

[Marius_Dragomir]

You can always edit that Data Table and split it by beat.hostname or if you want some specific names for each server, you can change the name filed inside the metricbeat.yml file on each server to whatever you want to describe that server and then split the Data Table by beat.name which is how that setting is shipped inside the logs.

[arun_prasath1 (Arun Prasath)]

In the metricbeat.yml, next to metricbeat.modules section there is line in the general section. so do I need to change the last line copied below to get the metric for hostname. how to split it by beat.hostname in the data table, could you please help me with some example or links about this.

metricbeat.modules:

#------------------------------- System Module -------------------------------

• module: system
  metricsets:
  • cpu
  • load
  • core
  • diskio
  • filesystem
  • fsstat
  • memory
  • network
  • process
    enabled: true
    period: 10s
    processes: ['.*']

#=============== General ========

The name of the shipper that publishes the network data. It can be used to group

all the transactions sent by a single shipper in the web interface.

#name:

The tags of the shipper are included in their own field with each

transaction published.

#tags: ["service-X", "web-tier"]

Optional fields that you can specify to add additional information to the

output.

#fields:

env: staging

output.elasticsearch:

Array of hosts to connect to.

[Marius_Dragomir]

@arun_prasath1

all the transactions sent by a single shipper in the web interface.

name:

You need to change this part. The name will show as beat.name field. beat.hostname is automatically generated from the network settings of that machine so whatever hostname you have set for that server it will be sent to Elasticsearch.

As for spliting the Data Table, after you edit the data table that I mentioned a few posts ago, you need to do the following steps:

1. Select "Add Sub-buckets"
2. Select "Split Table"
3. Select "Terms" as the Sub-aggregation
4. Select beat.hostname or beat.name as the Field and then click on the Run button.

This will show you a version of that table for each different server. It will look like this (i only have one server in this screenshot):

[arun_prasath1 (Arun Prasath)]

I was trying to create the new visualization to get the metrics, so could not see the metric options in y-axis. It seems there are many default metricbeat visualization in the dashboards in which there it exists options to split chart. This visualization I can use for detailed data report.
In the main page, I like to have the trend chart for the disk utilization(actual usage) for all my servers. I mean to have separate trend chart for all server to view the disk utilization. Is that possible in kibana ? I have to use the line chart for trend chart but how to get the graph for all the filesystems in one line chart. Any idea how I can achieve this.