Exclude empty values from output


#1

Hello
May be anybody know how can I exclude empty lines for disk usage stat? For example where average fs.total = 0


(ruflin) #2

We are working on a general filtering proposal (implementation not completed yet): https://github.com/elastic/beats/issues/451


#3

Wow, thanks for the reactive reply!


(Monica Sarbu) #4

Are you using Kibana to visualize the data? If the Beats are not exporting the zero value fields, it might be that Kibana will raise an error when displaying the widgets in the dashboard as it doesn't know the value of the missing field.


#5

Yes, Kibana.
My config is Elasticsearch + Topbeat + Kibana.
As I understood Ruflin's post after filtering implementation I will be able to configure Topbeat (or any other beat) to send only non_empty values into Elasticsearch, by next config

` filter:

  • drop_event:
    equals:
    "fs.total": 0`

The picture above is a good example, because the first 3 lines are from Windows machine and there is no device "a". Device "d" is a CD and it can or can not be empty, btw what will behavior of shipper in such case?

Regarding Linux machines:
It seems that Topbeat ships everything from Linux df. For example I don't need to know what is happening with my tmpfs or udev and other virtual fs, I need only know about /dev/sda1, /dev/sda2 etc. Is it possible to configure topbeat to send only necessary metrics?

Or maybe the better way is to configure Kibana to show only that I need?


(system) #6