How can I configure filebeat to only ship a percentage of logs (a sample if you will) In my application's log folder the logs are chunked to about 20 megs each. I want filebeat to ship only 1/300th of that log volume to logstash. Can filebeat filter a percentage of all log data? Haven't seen anyt…

Ship only a percentage of logs to logstash

warkolm (Mark Walkom) March 5, 2017, 7:23pm 2

Filebeat cannot, you may be able to with Logstash and something like the aggregation filter.

1 Like

Topic		Replies	Views
Can I use conditional logic to filter log lines with filebeat? Beats filebeat	4	1172	May 22, 2017
Enquiry about sampling in elasticsearch Beats filebeat	1	296	March 31, 2021
FileBeat - Ship only Filtered log lines Beats	6	1380	July 5, 2017
How to limit filebeat logs to logstash Beats filebeat	2	1941	September 15, 2017
Filebeat can't keep up with logs volume Beats filebeat	4	826	November 21, 2021

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

Ship only a percentage of logs to logstash

Related topics