Grok Filter Logstash Multiple Lines

Elastic Stack Logstash
1

I have some problem about the log's format that I had to parse. My logs have a format like this:
`==================================================================
Report : WARNING
Date : Thu Jun 18 16:52:54 2017
Description : Did not install signal handlers to cleanup resources.
Node : swim-host
Process : java <14517>
Thread : main thread 40ea6940
Internals : V6.3.130716OSS

Report : INFO
Date : Thu Jun 18 16:52:59 2017
........ .........`
and following like that. So in my case the separator between the single instances is the sequence of 88 "=". But grok filter considers by default as separator the end of line. So when I start logstash I have a grokparsefailure because it applies the grok filter, that I wrote for the entire instance, to just one line. For example if I try to parse a .log file whit just one instance of collected data and I start logstash I have 8 grokparsefailure, one for each line. I tried with the codec multiline or the gsub mutate filter but I couldn't solve the problem. How can I solve the issue?

2

Try this grok filter

^================================================================== 
Report : %{NOTSPACE:Report}
Date : %{DAY:day} %{MONTH:month} %{MONTHDAY:day} %{TIME:hour} %{YEAR:year}
Description : %{DATA:Description}.
Node : %{NOTSPACE:node}
Process : %{DATA:Process}
Thread : %{DATA:Thread}
Internals : %{NOTSPACE:Internals}

works on https://grokdebug.herokuapp.com/ with:

INPUT:

================================================================== 
Report : WARNING
Date : Thu Jun 18 16:52:54 2017
Description : Did not install signal handlers to cleanup resources.
Node : swim-host
Process : java <14517>
Thread : main thread 40ea6940
Internals : V6.3.130716OSS

PATTERN:

^================================================================== 
Report : %{NOTSPACE:Report}
Date : %{DAY:day} %{MONTH:month} %{MONTHDAY:day} %{TIME:hour} %{YEAR:year}
Description : %{DATA:Description}.
Node : %{NOTSPACE:node}
Process : %{DATA:Process}
Thread : %{DATA:Thread}
Internals : %{NOTSPACE:Internals}

RESULT:

    {
  "Report": [
    [
      "WARNING"
    ]
  ],
  "day": [
    [
      "Thu"
    ],
    [
      "18"
    ]
  ],
  "month": [
    [
      "Jun"
    ]
  ],
  "hour": [
    [
      "16:52:54"
    ]
  ],
  "HOUR": [
    [
      "16"
    ]
  ],
  "MINUTE": [
    [
      "52"
    ]
  ],
  "SECOND": [
    [
      "54"
    ]
  ],
  "year": [
    [
      "2017"
    ]
  ],
  "Description": [
    [
      "Did not install signal handlers to cleanup resources"
    ]
  ],
  "node": [
    [
      "swim-host"
    ]
  ],
  "Process": [
    [
      "java <14517>"
    ]
  ],
  "Thread": [
    [
      "main thread 40ea6940"
    ]
  ],
  "Internals": [
    [
      "V6.3.130716OSS"
    ]
  ]
}
1 Like
3

Can you post your configuration file? Also, are you wanting to group the message in between the = signs as one event or each line seperate?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.