Date Parse failuer

Hello All,

I am tying to create a index below are the config file and sample log. Please let me if anything wrong in my config file.
Thanks for your help!

Log :
2017-10-02 01:00:02,782 DEBUG [com._360commerce.foundation.util.DBConnection] SQL = SELECT VAL_TYPE.VALTYPEID, VAL_TYPE.VALIDATORNAME, VAL_TYPE.CLASS, VAL_TYPE.PACKAGE FROM PARM_VALIDATOR, VAL_TYPE WHERE PARM_VALIDATOR.VALTYPEID = VAL_TYPE.VALTYPEID AND PARM_VALIDATOR.PARMID = ?
2017-10-02 01:00:02,782 DEBUG [com._360commerce.foundation.util.DBPreparedStatement] PreparedStatment variables = Count = 1
Variables = 2541,

Config File:

input {
file {
path => "home/gprasad/btest.log"
#type => "LOG"
start_position => "beginning"
codec => multiline
{
pattern => "^\A%{TIMESTAMP_ISO8601}"
negate => true
what => previous
}

}
}

filter{

mutate
{
gsub => ["message", "\n", ""]
gsub => ["message", "\r", ""]
gsub => ["message", "\t", ""]
}
grok {
match => [ "message", "%{TIMESTAMP_ISO8601:Date} *%{LOGLEVEL:Type} [%{DATA:Application}] %{GREEDYDATA:Message}" ]
overwrite => [ "message" ]
}
date {
match => [ "Date","[yyyy-MM-dd HH:mm:ss,zzz]"]
target => "Date"
}
}
output
{
#stdout{}
stdout {codec => rubydebug}
elasticsearch
{
hosts => "localhost"
index => "kp-log-1296-bo-index1"
}
}

Error:

{
"path" => "home/gprasad/btest.log",
"Type" => "DEBUG",
"@timestamp" => 2017-11-02T09:32:57.906Z,
"Message" => "SQL = SELECT VAL_TYPE.VALTYPEID, VAL_TYPE.VALIDATORNAME, VAL_TYPE.CLASS, VAL_TYPE.PACKAGE FROM PARM_VALIDATOR, VAL_TYPE WHERE PARM_VALIDATOR.VALTYPEID = VAL_TYPE.VALTYPEID AND PARM_VALIDATOR.PARMID = ?",
"@version" => "1",
"host" => "oc1008401175.ibm.com",
"message" => "2017-10-02 01:00:02,782 DEBUG [com._360commerce.foundation.util.DBConnection] SQL = SELECT VAL_TYPE.VALTYPEID, VAL_TYPE.VALIDATORNAME, VAL_TYPE.CLASS, VAL_TYPE.PACKAGE FROM PARM_VALIDATOR, VAL_TYPE WHERE PARM_VALIDATOR.VALTYPEID = VAL_TYPE.VALTYPEID AND PARM_VALIDATOR.PARMID = ?",
"Application" => "com._360commerce.foundation.util.DBConnection",
"Date" => "2017-10-02 01:00:02,782",
"tags" => [
[0] "_dateparsefailure"
]
}

I have also used below date match format as well.

date {
match => [ "Date","[yyyy-MM-dd HH:mm:ss,SSS]"]
target => "Date"
}

Why do you have square brackets in your date pattern? The Date field doesn't contain any square brackets.

Thanks for your response.
I have removed the brackets and it worked.
Though we have passed this as Date, any idea why Kibana dosen't recognize this as date field. it still populate it as string.

When the square brackets were there ES autodetected the field as a string. Now that you've fixed your date filter the field it'll be detected as a date but existing mappings in an index will never change. Deleting the current index or waiting until the next index is created (perhaps tomorrow, depending on your configuration) will fix it.

Thanks a lot for your response

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.