Ingest pipeline: how to ignore log line based on condition

Elastic Stack Elasticsearch
1

Hello,

I am delivering logs from filebeat into ES via Ingest pipeline.

I want to drop log lines based on some condition (e.g. if (httpcode != 200) then ignore). Something like drop filter in LS.

How can I achieve this?

Thanks.

2

There's no drop functionality at this stage, you can do it in filebeat though - https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#exclude-lines

3

Yes, but filebeat has no ability to parse line against regexp (to extract filter field).

Look at Exported fields in filebeat processors
which suggests to use ingest instead of filebeat :slight_smile:

I think it would be nice (and easy) to implement an ability to ignore a line in ingest depending on some condition (after parsing a line against regexp).

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.