Ingest pipeline: how to ignore log line based on condition

Hello,

I am delivering logs from filebeat into ES via Ingest pipeline.

I want to drop log lines based on some condition (e.g. if (httpcode != 200) then ignore). Something like drop filter in LS.

How can I achieve this?

Thanks.

There's no drop functionality at this stage, you can do it in filebeat though - https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#exclude-lines

Yes, but filebeat has no ability to parse line against regexp (to extract filter field).

Look at Exported fields in filebeat processors
which suggests to use ingest instead of filebeat :slight_smile:

I think it would be nice (and easy) to implement an ability to ignore a line in ingest depending on some condition (after parsing a line against regexp).

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.