Two different date formats from logs. Handle with one multiline.pattern?

Elastic Stack Beats
1

Hi,

I'm trying to match two different time formats within different application logs (two out of five use ISO8601, the other three DD-MM-YYYY HH:mm:ss,mmm.

I figured I'd be able to solve it like this:

  multiline.pattern:             '^${timestamp}'
  multiline.negate:              true
  multiline.match:               after
  
  encoding:                      iso8859-1

  patterns:
    timestamp:                   '$(${timestamp1}|${timestamp2})'
    # capture dates of type '2017-01-01 01:02:03.456'
    timestamp1:                  '^(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{3}'
    # capture iso8601 datetime '2021-11-25T14:40:07.5582771+01:00|'
    timestamp2:                  '^(?>\d\d){1,2}-(?:0?[1-9]|1[0-2])-(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])[T ](?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)(?:Z|[+-](?:2[0123]|[01]?[0-9])(?::?(?:[0-5][0-9])))'

But this generates an error:

2021-11-29T09:11:55.009+0100 ERROR instance/beat.go:989 Exiting: Failed to start crawler: starting input failed: Error while initializing input: cyclic reference detected for key: 'timestamp' accessing 'filebeat.inputs.0.multiline.pattern' (source:'filebeat.yml')
Exiting: Failed to start crawler: starting input failed: Error while initializing input: cyclic reference detected for key: 'timestamp' accessing 'filebeat.inputs.0.multiline.pattern' (source:'filebeat.yml'

Any ideas as to why the intended solution doesn't work? Or am I going at this from the wrong perspective?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.