Filebeat: Match literal * or ** in messages

[Krog (Ken Rogers)]

I have log files that contain the following:

<15:00:51.504 DBG ...
<15:00:52.173 *WRN* SM ...
<15:00:52.173 **ERR** TSS ...
<15:00:52.173 TRC EWF ...

I would like to drop any messages that do not contain WRN or ERR from filebeats, but I can't figure out how to format my regular expression to do that.

When I use this:

processors:
  - drop_event:
      when:
        not:
          regexp:
            message: " *WRN* | *\*ERR*\* "

I get an error about the unknown escape sequence. If I use single quotes:

processors:
  - drop_event:
      when:
        not:
          regexp:
            message: ' *WRN* | *\*ERR*\* '

I never get any output to logstash.

How can I match both *WRN and ERR successfully?

Best Regards
Ken

[Debashis (Debashis Mondal)]

Hi @Krog,

As per my understanding, you want to drop any message that does not contain WRN or ERR.
Hence it implies that you just want to send the WRN and ERR messages to logstash.

you can do it by configuring "include_lines" option

filebeat.prospectors:

• input_type: log
  include_lines: ["WRN", "ERR"]

[Krog (Ken Rogers)]

Great - Let me try that and hopefully it will work better than trying to use a processor section.

Ken

[Debashis (Debashis Mondal)]

Hello @Krog,

If you want to escape * then use \\* . I think it will help you more.

[Krog (Ken Rogers)]

That's what I ended up doing in the prospectors section:

- type: log
  paths:
    - /Images_Prod/Server1/logs/*
  exclude_files: ['\.gz$']
  include_lines: [' \*WRN\* ',' \*\*ERR\*\* ']

My original thought was to use a processor section but I couldn't figure out how to escape the asterisks and avoid the dreaded unknown escape sequence error.

Your suggestion to use the include_lines directive was a much better solution anyway.

Thank you for the assist.

Regards

Ken