Options to compress topbeat index

Elastic Stack Beats
1

Hi Team,

We have topbeat agent running on 10 systems and 10 second pulse.
It is further configured for process, CPU and system logs. the average payload per 10s is 150KB ~300KB resulting in 300 MB per server per day resulting 3GB+ data per day.

I understand that the indexes are already compressed . Are there any other options to further reduce the index size...

Note: I am aware of archival backup and restore but not intending to implement due to project requirements.
Also do we have benchmarks of topbeat compared to other agents in terms of data, resource utilization etc. Kindly point if any

Look forward to suggestions.

Regards,

2

ES compresses by default, but take a look at https://www.elastic.co/guide/en/elasticsearch/reference/2.4/index-modules.html#index-codec

3

If you are using elasticsearch 5.0 with metricbeat the events will take less space as scaled_floats were introduced which are especially useful for percentage values.

4

This topic was automatically closed after 21 days. New replies are no longer allowed.