Why Filebeat is not picking up multiline ISO timestamp pattern but picking when pattern is defined manually?

[elasticheart]

Hi,

I have log lines like below;

[2020-11-11T11:19:00+05:30] [INFO] [msg1] [msg2] [msg3] [msg4] [msg5] [msg6] [msg7] [msg8] [msg9]

The msg9 attribute may contain multi-line message, so I have decided to configure multi-line pattern in filebeat.yml like;

multiline.pattern: '^\[%{TIMESTAMP_ISO8601}\] '
multiline.negate: true
multiline.match: after

But, this was not working because I was getting all the log lines in a single message. Then, I gave the pattern manually, like below;

multiline.pattern: '^\[[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}T[[:digit:]]{2}:[[:digit:]]{2}:[[:digit:]]{2}\+[[:digit:]]{2}:[[:digit:]]{2}\]'

This worked like charm, and every log line gets out as separate messages. I was just wondering, why the TIMESTAMP_ISO8601 pattern configuration is not working?

Thanks.

[Kaiyan_Sheng]

Hmm by looking at the timestamp example under multiline documentation, seems like the recommended way is not to use TIMESTAMP_ISO8601.
Here is the example:

multiline.type: pattern
multiline.pattern: '^\[[0-9]{4}-[0-9]{2}-[0-9]{2}'
multiline.negate: true
multiline.match: after

[elasticheart]

Thanks @Kaiyan_Sheng