Filebeat multiline is not working as expected

[Abu_Tahir]

The below set of lines are my test log

[2016-08-24 11:49:14,389] Started new event
[2016-08-24 11:49:14,395] Content of processing something
[2016-08-24 11:49:14,399] End event
[2016-08-24 11:49:14,389] Started new event
[2016-08-24 11:49:14,395] Content of processing something
[2016-08-24 11:49:14,399] End event

my conf

- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /Users/mac/logs/*.log
  multiline.pattern: '^*Started new event'
  multiline.negate: false
  multiline.match: after
  multiline.flush_pattern: '^*End event'

I am still seeing all the above logs are still going as individual logs . Not sure what am i doing wrong here

[Tek_Chand]

@Abu_Tahir,

The log you have provided above are seems already in single line logs because every line is started with date and time. Thats why you are getting these logs as single line logs.

Can you please elaborate more what you want to achieve?

Thanks.

[steffens (Steffen Siering)]

I don't think you need flush_pattern at all.

You regex does not match you contents. I don't think ^* is even valid, is this would mean something like "any number of line beginnings".

The multiline.match docs give some idea how the configs work: https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html

you patterns is like bacbdcbec, which you want to split into bac, bdc, bec. This can be achieved via:

multiline.pattern: 'Start new event'
multiline.negate: true
multiline.match: after