Grok multiline coded help

Elastic Stack Logstash
1

Hi there,

Could you please help me to create a logstash grok pattern using multiline codec for parsing cobbler logs?

At times, the logs span over multiple lines

Sample log:

Mon May  7 05:53:49 2018 - DEBUG | get_items; ['system']
Mon May  7 05:53:49 2018 - DEBUG | done with get_items; ['system']
Mon May  7 05:53:49 2018 - WARNING | warning: kernel option length exceeds 255
Mon May  7 05:53:49 2018 - WARNING | warning: kernel option length exceeds 255
Mon May  7 05:53:49 2018 - WARNING | warning: kernel option length exceeds 255
Mon May  7 05:53:49 2018 - WARNING | warning: kernel option length exceeds 255
Mon May  7 05:53:50 2018 - INFO | generating: /var/lib/tftpboot/pxelinux.cfg/test
Mon May  7 05:53:50 2018 - INFO | generating: /var/lib/tftpboot/grub/TEST
Mon May  7 05:53:50 2018 - INFO | Writing template files for test
Mon May  7 05:53:50 2018 - INFO | REMOTE find_items_paged(system); criteria({}); sort(name); user(?)
Mon May  7 05:53:50 2018 - INFO | find_items; ['system']
Mon May  7 05:53:50 2018 - INFO | REMOTE version; user(testuser)
Mon May  7 05:53:54 2018 - INFO | REMOTE generate_kickstart; user(?)
Mon May  7 05:53:54 2018 - INFO | generate_kickstart
Mon May  7 05:53:54 2018 - ERROR | 
# *** ERROR ***
#
# There is a templating error preventing this file from rendering correctly. 
#
# This is most likely not due to a bug in Cobbler and is something you can fix.
#
# Look at the message below to see what things are causing problems.  
#
# (1) Does the template file reference a $variable that is not defined?
# (2) is there a formatting error in a Cheetah directive?
# (3) Should dollar signs ($) be escaped that are not being escaped?
#
# Try fixing the problem and then investigate to see if this message goes
# away or changes.
#
# 
# 'bonding'
#   File "/usr/lib/python2.7/site-packages/cobbler/templar.py", line 208, in render_cheetah
#     data_out = t.respond()

I have tried using below snippet in my logstash conf file:

filter {
  if [path] =~ "cobbler.log" {
    grok {
        match => { "message" => "%{DATA:logDatetimeText} - %{WORD:logLevel} \| %{GREEDYDATA:logMessage}" }
    }
    date {
        match => [ "logDatetimeText", "EEE MMM  d HH:mm:ss YYYY", "EEE MMM dd HH:mm:ss YYYY" ]
    }

}

This gives me a _grokparsefailure when it encounters logs spanning multiple lines. So I'm looking at using s multilines code that negates all logs not matching the above specific date pattern. Can someone help me with this please?

2

I would suggest

input { stdin { codec => multiline { pattern => "^(Mon|Tue|Wed|Thu|Fri|Sat|Sun)" negate => true what => "previous" auto_flush_interval => 3 } } }
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.