Filebeat multiline patterns not working

(lalu) #1

filebeat v1.2.2 (64 bit)
OS: centos 6.7

multiline: --> patterns: --> '-' --> pattern: (not working)
multiline: --> pattern: (working)

Found detail here:
(Enhanced LS Config)

Is this supposed to work or feature not available yet.

(ruflin) #2

Multline support is part of the 1.2 release. The docs can be found here:

I'm not sure what you mena with your patterns / pattern above?

(Steffen Siering) #3

please add some sample logs you want to merge + regex you've configured. I can't even tell what you're trying to do.

(lalu) #6

(Steffen Siering) #7

You've got text instead of pictures. Here is a script to test your regexes, but I'm not too keen to type out all these pictures.

Put text in between 3 backticks '`' or use the </> button on inserted text in order to code-format your text.

(lalu) #8

I just want to know that the feature is available in Filebeat? The feature I mean more then one pattern in multiline.

(lalu) #9

Thanks Steffen for your trick for formating.

  pattern: '^Encore.DMS.Svc-'
  negate: true
  match: after

This above config works for me for single condition, but not sure about more then one condition like I wrote below.

      pattern: '^Encore.DMS.Svc-'
      negate: true
      match: after
      pattern: '^Encore.DMS.Svc-\.*RESPONSE:'
      negate: false
      match: after

(Steffen Siering) #10

multiple conditions are not supported yet. But one can use the | (OR-)regex operator in in pattern.

(lalu) #11

Log content in proper format:

Encore.DMS.Svc-6.2-0-1000013856-INFO|08:08:30:246|AgentFirstName : 
<s:Envelope xmlns:s="">
  <s:Body ...>
    <h2h-dms-request xmlns=...>
<soapenv:Envelope ...>
  <s:Header xmlns:s=... />
    <xrsi:h2h-dms-reply xmlns:xrsi=...>
Encore.DMS.Svc-6.2-0-1000013856-INFO|08:08:30:730|Return value:
Encore.DMS.Svc-6.2-0-1000013856-INFO|08:08:30:731|Code : 000

(lalu) #12

Thanks for your help.
I already have this config and it is working as expected.

  pattern: '(^Encore.DMS.Svc-)|(^Encore.DMS.Web-)|(^Encore.DASMonitor-)'
  negate: true
  match: after

But for my last condition I need to negate inside the regex. Which I think is not allowed or will not work because negate handled in separate line like "negate: true".
"! (^Encore.DMS.Svc-\.*RESPONSE:)"
Something like above can be doable or any other way this can be achieved in Filebeat?

(Steffen Siering) #13

Do I understand you correctly? You just want to merge REQUEST plus RESPONSE attributes?

Check out this solution:
Regex is ^([[:space:]]|<|(.*\|){2}RESPONSE)
First sub-term matches a string starting with whitespace, the second sub-term matches a line starting with < and the third subfilter matches ... | ... |RESPONSE without even checking the content of the columns.

(system) #14