Grok multi-line mode

Elastic Stack Logstash
1

I am using (?ms) in my grok filter, but got an error (see RegexpError: undefined).
What should be the right way to lookup multiple lines using grok? in regular regex i amd doing

(?sm)(?<starttime>[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3} |[A-Z][a-z]{2}\s[0-9]{2},\s[0-9]{4}\s[0-9]{2}:[0-9]{2}:[0-9]{2}\s(AM|PM)|)
(\s|)(?<loglevel>[A-Z]{4,6})(\s|)(?<service>)(\s|)
(?<data>\s(.*?)(?=[\r\n]+\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}|\Z))
2

can you provide a sample input and your expected output?

3

Multiline should start with (?m), without s. You can use grok patterns.

The syntax for a grok pattern is %{SYNTAX:SEMANTIC}
The SYNTAX is the name of the pattern that will match your text.

As Sunile already asked, a sample please

4

Also the blog article might be useful for grok patterns.

5

this is an example of the log

2010-06-19 02:26:05,556 INFO [main] [${sys:switchMode}] onEntry/onExit: com.example.wsw on Port [0]
2010-07-23 14:58:47,579 DEBUG [main] [${sys:switchMode}]
  c.a.s.c.ClickerController:
  {POST [/api/devices/clicker/{portIndex}/move-clicker]}: moveClicker(int,Map)
  {GET [/api/devices/clicker]}: Clickers()
  {UPDATE [/api/devices/clicker/{portIndex}/move-clicker]}: moveClicker(int,Map)
  {DELETE [/api/devices/clicker]}: Clickers()
2010-07-23 14:58:47,579 DEBUG [main] [${sys:switchMode}]
  c.a.s.c.ClickerController:
  {POST [/api/devices/clicker/{portIndex}/move-clicker]}: moveClicker(int,Map)
  {GET [/api/devices/clicker]}: Clickers()
  (WeirdLocalDataInspector)
6

still failed to catch multiline. is it possible to give an example with my log?

2010-06-19 02:26:05,556 INFO [main] [${sys:switchMode}] onEntry/onExit: com.example.wsw on Port [0]
2010-07-23 14:58:47,579 DEBUG [main] [${sys:switchMode}]
  c.a.s.c.ClickerController:
  {POST [/api/devices/clicker/{portIndex}/move-clicker]}: moveClicker(int,Map)
  {GET [/api/devices/clicker]}: Clickers()
  {UPDATE [/api/devices/clicker/{portIndex}/move-clicker]}: moveClicker(int,Map)
  {DELETE [/api/devices/clicker]}: Clickers()
2010-07-23 14:58:47,579 DEBUG [main] [${sys:switchMode}]
  c.a.s.c.ClickerController:
  {POST [/api/devices/clicker/{portIndex}/move-clicker]}: moveClicker(int,Map)
  {GET [/api/devices/clicker]}: Clickers()
  (WeirdLocalDataInspector)
7

What does one of your multi-line events look like if you use

output { stdout { codec => rubydebug } }

Are you using a multiline codec to combine all the lines from a single log entry into one event?

8

sorry for the late response. I did not know about codec =>mutline. I was able to read multiple line using

input {
    file {
        path => "/usr/logs/*.log"
        codec => multiline {
            pattern => "^%{TIMESTAMP_ISO8601}"
            negate => true
            what => "previous"
        }
    }
}

Adding this here in case someone from the future sees this

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.