Logstash - grokparsefailure

Elastic Stack Logstash
1

Hi,

I am using Logstash 6.2.4 and my system is windows 10.
I am getting grokparsefailure when I add TIMESTAMPISO8601 field in grok filter.
This is my config file for logstash:

input {
file {
path => ["D:/TestLogs/*"]
start_position => "beginning"
sincedb_path => "/dev/null"
codec => multiline {
pattern => "^(\s)"
what => "previous"
}
}
}

filter {
grok {
match => {"message" => "%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{GREEDYDATA:message}"}
}
}

output {
stdout {}
}

This is my log:

10-05-2018 00:00:00.0031 DEBUG Thd: 6540 [Schedule] (checkSchedules) Chequea a: 10/05/2018 12:00:00 AM ON 10/05/2018 12:00:00 AM
10-05-2018 00:00:00.0031 DEBUG Thd: 6540 [Schedule] (checkSchedules) ThreadFinished: 00:00:00
10-05-2018 00:00:01.1904 ERROR Thd: 6568 192.168.56.1 [ManagedObject] (EntityLoad) Object not be loaded 192.168.56.1 AGENT

This is the output in logtstash command prompt:

      "path" => "D:/TestLogs/tm20180510",
      "host" => "BALP-SunilS",
   "message" => "10-05-2018 00:00:00.0031\tDEBUG\tThd: 6540\t\t[Schedule]\t(checkSchedules)\tChequea a: 10/05/2018 12:00:00 AM ON 10/05/2018 12:00:00 AM\r",
  "@version" => "1",
"@timestamp" => 2018-05-21T10:26:19.950Z,
      "tags" => [
    [0] "_grokparsefailure"
]

}

Can anybody help me to solve this issue.

2

TIMESTAMP_ISO8601 has year-month-day. You have month-day-year. It does not match. If that is all the parsing you want to do, then dissect is going to be a lot faster than grok.

dissect { mapping => { "message"=> "%{ts} {%{+ts}     %{message}" } }
3

@Badger

I have used dissect but it is taking some text also after time.

This is my contents of log

23-05-2018 00:00:27.7937 INFO Thd: 5268 [MiddlewareNet](b__166_0) AliveMessage sent to : BANGDEV (10.10.18.203) OK
23-05-2018 00:02:27.7943 INFO Thd: 7924 [MiddlewareNet](b__166_0) AliveMessage sent to : BANGDEV (10.10.18.203) OK
23-05-2018 00:04:27.7940 INFO Thd: 8200 [MiddlewareNet](b__166_0) AliveMessage sent to : BANGDEV (10.10.18.203) OK

This is my configuration file

input {
file {
path => ["D:/TestLogs/tm*"]
start_position => "beginning"
sincedb_path => "/dev/null"
}
}

filter{
dissect {
mapping => { "message"=> "%{ts} %{+ts} %{message}" }
}
}

output {
stdout { codec => rubydebug }
}

The output which i got is shown below:

{
"@version" => "1",
"ts" => "23-05-2018 00:02:27.7943\tINFO",
"path" => "D:/TestLogs/tm20180510",
"@timestamp" => 2018-05-23T09:22:45.475Z,
"message" => "\tThd: 7924\t\t[MiddlewareNet]\t(b__166_0)\tAliveMessage sent to : BANGDEV (10.10.18.203) OK\r",
"host" => "BALP-SunilS"
}
{
"@version" => "1",
"ts" => "23-05-2018 00:00:27.7937\tINFO",
"path" => "D:/TestLogs/tm20180510",
"@timestamp" => 2018-05-23T09:22:45.401Z,
"message" => "\tThd: 5268\t\t[MiddlewareNet]\t(b__166_0)\tAliveMessage sent to : BANGDEV (10.10.18.203) OK\r",
"host" => "BALP-SunilS"
}

Can you please explain why it is taking INFO also along with time.
Actually 5 spaces are there between time and INFO. But when i copied the text here, it is taking one space.

4 
"ts" => "23-05-2018 00:00:27.7937\tINFO",

The \t indicates that the separator is tab, not a group of spaces. So you need to use a tab in the dissect filter.

5

@Badger

May I know how to use tab in dissect filter.

6

The character between %{+ts} and %{message} should be a tab, not a space.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.