Multiline problem with 2 "first" lines

Elastic Stack Beats
1

I just implemented a new ELK server to centralise all our servers' logs so I'm still pretty new to the configuration and I'm having quite a puzzling problem.

I am trying to get multilines log to be caught as one event and it works pretty well for most of them but some of them are tricky and have two "first" lines and what happen is that one event is sent for the first "first" line and then a second event is sent for the second "first" line and the rest of the multilines log.

I'm trying to find the right pattern to caught all of it in one event but I can't figure it out so I turn myself to you for help.

Here is an example of log we have:

[Aug-03 01:16:11.847].[ERROR] [http-nio-8443-exec-94-180] exceptions.AbstractExceptionMapper::errorResponse - Catching
javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: could not execute statement
    at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1763) ~[hibernate-entitymanager-4.3.11.Final.jar:4.3.11.Final]          
    at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1677) ~[hibernate-entitymanager-4.3.11.Final.jar:4.3.11.Final          
    at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1683) ~[hibernate-entitymanager-4.3.11.Final.jar:4.3.11.Final]

See how the [ERROR] tag is on one line but the rest of the log is below that second line, so when doing a search in Kibana for ERROR, it's not working. So what I'm trying to do is to send all of this as one event. Also the second line can change, it's not always the same message.

My current multiline pattern is as follow

multiline.pattern: '^\t|^[[:space:]]+(at|...)|^Caused by:'
multiline.negate: false
multiline.match: after

Is there a pattern for what I'm looking for?

2

How about using the timestamp at the pattern? https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html#_timestamps

3

And change the negate to true to catch all the lines after it? As I said, still pretty new to this so I don't understand all the configurations yet. I'm trying things in a sort of regex playground, it seems to work.

I'll give it a try and see what are the results.

Thanks!

4

Yep, just like the example shows, but customize the date pattern.

5

It worked perfectly!

Thanks a lot!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.