Metricbeat module: collect metric only if there is a change

[annk]

Hi ,

I am new to beats , I was wondering if there are any options in metricbeat modules which I can turn on to collect metrics only if there is a change during the interval ?

are there any way I can avoid repetitions like these ..

          Time 	           system.filesystem.device_name      	system.filesystem.available
    May 4th 2018, 11:41:06.503	   /apps	                                72.243GB
May 4th 2018, 11:40:06.503	   /apps	                                72.243GB
May 4th 2018, 11:39:06.503	   /apps	                                72.243GB
May 4th 2018, 11:38:06.503	   /apps	                                72.243GB
May 4th 2018, 11:37:06.503	   /apps	                                72.243GB
May 4th 2018, 11:36:06.503	   /apps	                                72.243GB
May 4th 2018, 11:35:06.503	   /apps	                                72.243GB

(I got error when I tried to upload a screenshot of my discover tab)

Thanks
Anu

[andrewkroh]

There's nothing like that in Beats. Generally Beats push raw data to an output destination and any follow-on processing or downsampling is done at the server.

I think you could accomplish what you asked in Logstash with an aggregate filter that memorizes the previous value and if they are the same it calls event.Cancel().

And in Elasticsearch + X-Pack 6.3 a new feature called rollups is being introduced. This would allow you to create a rollup index that contains downsampled metrics leading to small storage requirements when retaining metrics for long periods.