Multiple files with different Multilnes

Elastic Stack Beats
1

I am attempting to read 2 files with multiline logs. But I am not getting the desired result.
In test1.log line starts with date.
In test2.log line starts with time.
I get results for each log individually but when I combine them I don't get correct results.

filebeat.prospectors:

  • type: log
    enabled: true
    paths:

    • /gen_test1/test1.log
      fields:
      Region: AMS
      multiline.pattern: '^(\d{4}-\d{2}-\d{2})'
      multiline.negate: true
      multiline.match: after

  • type: log
    enabled: true
    paths:

    • /gen_test2/test2.log
      fields:
      Region: EMEA
      multiline.pattern: '^(\d{2}:\d{2}:\d{2}.\d{3})'
      multiline.negate: true
      multiline.match: after
2

Actually the yml above is working.
But when I use multiple multiline patterns using "or" like this, it gives result only for test2.log.

filebeat.prospectors:

  • type: log
    enabled: true
    paths:

    • /gen_test1/test1.log

    fields:
    Region: AMS
    multiline.pattern: '^((\d{4}-\d{2}-\d{2})'|(\d{2}:\d{2}:\d{2}.\d{3}))'
    multiline.negate: true
    multiline.match: after

  • type: log
    enabled: true
    paths:

    • /gen_test2/test2.log

    fields:
    Region: EMEA

    multiline.pattern: '^((\d{4}-\d{2}-\d{2})'|(\d{2}:\d{2}:\d{2}.\d{3}))'
    multiline.negate: true
    multiline.match: after

3

Hi @astrovj and welcome :slight_smile:

When posting commands or configuration you can use the </> button to monospace it so it is easier to read.

Regarding your configuration, if they are exactly the ones you are using, I think that the quote before the | (the "or") should be removed.

4

Thanks for your reply. I missed removing that quote before the |.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.