[ASK] Other log collector or log sender

[merceskoba]

Hello Guys
Do you know other log collector such as Filebeat ?
Filebeat doesn't has sampling_rate(https://www.scalyr.com/help/scalyr-agent#filter)

My company doesn't want using Scalyr anymore. That's so expensive
That's why i am trying to make POC using Filebeat as log collector or log sender.
Do you know guys, other log collector or log sender app ?
Please help me out of this POC

[Christian_Dahlqvist]

Filebeat allows you to drop logs based on patterns - might that work for you? That way you are sure you are not sampling out any vital logs.

[merceskoba]

Filebeat can drop logs based on patterns using exclude or include right ?
But, the case is when you need only 100 logs of 1000 logs. They are same logs.
For example, in my company case we are using Golang and sometimes if there is a error, it will write like this
account_gen.go ERR dpkg/blablablabla/ [TIMESTAMP]
account_gen.go ERR dpkg/abcbacbacb/ [TIMESTAMP]
account_gen.go ERR dpkg/pqpqpqpqp/ [TIMESTAMP]
untill the thousands line.
With sampling_rate 0.1 (scalyr), i can send only 0.1*TOTAL_AMOUNT logs.
{ match_expression: "go", sampling_rate: 0.1 } means, i will allow "go" word and the sampling_rate is 0.1 or 10% of total logs.
if i use include_lines it means i allow all. If i use exclude_lines, it means i disallow all.
sampling_rate: 0 means exclude_lines
sampling_rate: 1 means include_lines
sampling_rate: 0.1 or 0.2 or 0.3 means include_lines but only 0.1 (10%) of total logs

That's why we really need sampling_rate, so developers know there are some errors in some logs without sending all logs.
I trust Filebeat because yeah, you are always updating it dude...
but still i need sampling_rate... if you have suggestions, let me try for my POC