Can we add fields value based on regex patterns in include_lines for the same log

I am parsing a log file which has error from 3 different process.
I have added the input file in path and have added the 3 regex pattern in include line unique to 3 processes.
Can I add field column which will be populated with the log line based on the regex pattern it matched in include_lines tag?


logs are:
[p1_err] adc....
[p2_err] adc....
[p3_warn] adc....
[p1_warn] adc....
[p3_err] adc....

in my filebeat.yml


  • type: log
    • <log_file>
      include_lines: ['p1_err', 'p2_err', 'p3_err']

I want to see the count of p1_err,p2_err,p3_err lines in output on kibana.

include_lines is just for prefiltering, it's no parser.

You can either go schemaless and use filtering on the "message" field in kibana UI in order to count events matching a pattern or use grok (Logstash or Ingest Node), or dissect (Filebeat or Logstash or Ingest Node) to do some minimal parsing. E.g. see Filebeat dissect docs.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.