Filebeat Preprocess

Elastic Stack Beats
1

Hi All,

I've got some logs with sensitive data in them that I'd like to exclude before shipping them. I know I can remove entire lines, but the sensitive data is just a portion of the line, the rest of which I need.

Is there a way to modify input/output in such a way to allow this?

2

Filebeat does not process the log lines, so this is a task that has to be done in Logstash. I understand that this means having to send the data across the wire, as long as LS is not installed on the same machine.

One potential solution is using JSON as log format. The next major version of Filebeat can read JSON log lines and there you can use the filters in case the sensitive data is a in a specific field.