Filebeat processors

Elastic Stack Beats
1

Hi everyone,

I have a question about the filebeat processors (extract_array, drop_event, drop_fields).
My filebeat agent collects about 2500 logs lines a second. Do you think that using these processors can lead to huge CPU usage ? And if it's possible, what about delegate these operations to logstash ?

Thanks for your answer.
Khaled

2

Are you already sending the data to Logstash? If yes, then I would leave any data transformation to be done in Logstash.

If not, you will need to try, but I don't think that these processor will impact the CPU usage in any way.

3

Hi Leandro,
I agree with you to do data tranformation in logstash. My filebeat process sends logs to a kafka server and then logstash consume the kafka messages. I'm facing a huge CPU usage when filebeat collects logs. So i'm gonna try to move filebeat data transformation in logstash and i will see if this decrease the CPU usage.
Thx for your help.

Khaled

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.