Running Elastic Stack 7.10.2 ( Filebeat, Elastic search). The input for logs in filebeat is cloudfoundry (6.5).
The filebeat runs as a container in cloud foundry.
We need to read multiline messages coming from cloudfoundry hosted applications. The pattern of data coming in cloud foundry is like below. Every message from the Application is between marker BEG and END.
2021-05-27T17:36:02.259+05:30 [APP/WEBR/MOB/2] [OUT] BEG 2021-07-27 11:36:02 INFO ntered into getCategoriesByCode >>>>>>>>>>> END
2021-05-27T17:36:02.260+05:30 [APP/WEBR/MOB/2] [OUT] BEG 2021-07-27 11:36:02 ERROR Invalid input. Number of categories should be between 1 and 5 java.lang.ArithmeticException: / by zero
2021-05-27T17:36:02.260+05:30 [APP/WEBR/MOB/2] [OUT] at com.xm1(CommonMessageController.java:51)
2021-05-27T17:36:02.260+05:30 [APP/WEBR/MOB/0] [OUT] at jdk.internal.reflect.GeneratedMethodAccessor58.invoke(Unknown Source)
2021-05-27T17:36:02.260+05:30 [APP/WEBR/MOB/0] [OUT] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) END
Below pattern is configured in filebeat
multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}[[:alnum:]][0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}\+[0-9]{2}:[0-9]{2}[[:space:]]\[[[:alpha:]]{3}/[[:alpha:]]{4}/[[:alpha:]]{3}
/[0-9]\][[:space:]]\[OUT\][[:space:]]BEG'
multiline.negate: true
multiline.match: after
The same is not working and multilines are not getting clubbed. So we extracted the same log from cloudfoundry app to flat file and rerun the same after chaning the input to file and it worked perfectly.
Is this Cloudfoundry issue ? How we can resolve it.