How to use multiline to break up messages delimited with - (hyphen)

Elastic Stack Logstash
1

So I am going through this one type of log where some of the messages are broken up by hyphen (-----) for example:

    2015-05-01 10:08:08 [http-request] Level Outbound Message
    ------------------------
    ID: 2
    Address: site address
    Encoding: UTF-8
    <soap: Envelope ...>
      <soap: Body>
        <...>
    </soap:...>
    -------------------------------------------

I notice right after the message says "outbound" or "inbound" the starting set of hyphens appear followed by the close hyphens. Yes the starting hyphen line is shorter then the closing.

My question is how would you go about using multiline on this type of log? Its not like a stack trace example. The only pattern I notice is the hyphens. Is there a if this amount of hyphens occur add next line until next set of longer hyphens appear? Thanks.

Mike

4

Hi Mike,
I think you might have guessed it out by now. If not, i think if i understand your question, you would want to convert the entire multilines into one event using "multiline filter". If that is the case, then you can ask the filter to say that if any of the line does not start start with a YEAR (%{YEAR}) consider it to be a part of the same event.

For ex :
> multiline {

                        pattern => "^%{YEAR}"
                         negate => true
                         what => previous
          }

I hope i have answered your doubt. Apologies, if that is not what is intended.

5

Chaitanya, I figured it out in theory that is exactly what I did. Just some really complex pattern that is generic with most of the log and then negate. So If anyone else is trying to figure this out you can just negate the similar patterns that are not multiline. Thanks for reminding me about this problem almost forgot about it after working it through.