I have some logs i want to ship to elastic.
the format is quite simple
12.12.2016 10:57:11: OCR for ActID=4618887 ..\n (2 pages 9,14s)
The Match-Pattern looks like this (but i have allready tried %{DATESTAMP} instead of the numbermatching at the start
match => { "grokmessage" => "(?(\d{2}.\d{2}.\d{4} \d{2}:\d{2}:\d{2}))([:|A-Z|a-z| |=])%{NUMBER:activityid}(( |.)+)\n +(%{NUMBER:pagecount}([:|A-Z|a-z| |=])(?([0-9]+,?[0-9]+))(((.)+))" }
The full filter is:
filter {
if [fields][application] == "ocr" {
mutate {
add_tag => [ "OCR_grok_entered" ]
}
mutate { add_field => { "grokmessage" => "%{message}" } }
grok { match => { "grokmessage" => "(?<eventtime>(\d{2}.\d{2}.\d{4} \d{2}:\d{2}:\d{2}))([:|A-Z|a-z| |=]*)%{NUMBER:activityid}(( |\.)+)\\n +\(%{NUMBER:pagecount}([:|A-Z|a-z| |=]*)(?<timeelapsed>([0-9]+\,?[0-9]+))(((.)+))" } }date {
match => ["eventtime", "dd.MM.yyyy HH:mm:ss"]
target => "@timestamp"
}
}}
when i try the config i get following output
{
"@timestamp" => 2016-12-12T09:57:21.392Z,
"offset" => 38799,
"@version" => "1",
"input_type" => "log",
"beat" => {
"hostname" => "S0001",
"name" => "S0001",
"version" => "5.0.2"
},
"host" => "S0001",
"source" => "L:\OCR\it20161212.log",
"message" => "12.12.2016 10:57:11: OCR for ActID=4618887 ..\n (2 pages 9,14s)",
"fields" => {
"application" => "ocr",
"service" => "ocrservice"
},
"type" => "ocrlog",
"grokmessage" => "12.12.2016 10:57:11: OCR for ActID=4618887 ..\n (2 pages 9,14s)",
"tags" => [
[0] "beats_input_codec_plain_applied",
[1] "OCR_grok_entered",
[2] "_grokparsefailure"
]
}
What would be the best way to debug this _grokparsefailure.
somehow i'm hitting a wall here, the pattern works fine in grokconstruktor
Its also working fine in grokdebugger
something i'm doing wrong but i don't know what 