Date is not being parsed with AM / PM

[shani]

HI, this is my config file. I want to extract date from the log and want to create two field like

• time_mentioned_in_log
• time_when_log_was_received_at_logstash

Raw Log:

3/2/2020 10:14 AM TYPE=Information USER= COMP=ABCXYZ.local SORC=Dummy CATG=(0) EVID=1 MESG=some_msg_here

configuration file:

input {

file {
path => "/etc/logstash/files/time.log"
start_position => beginning
sincedb_path => "/dev/null"
}
}

filter {
date {
match => ["message", "d/M/YYYY HH:mm a" ]
target => "logtimestamp"
}
}

output {
stdout { codec => rubydebug }
}

Please help me.

[Badger]

The pattern in the date filter has to consume the entire field, so for that message you would have to use

date { match => [ "[message]", "d/M/YYYY HH:mm a' TYPE=Information USER= COMP=ABCXYZ.local SORC=Dummy CATG=(0) EVID=1 MESG=some_msg_here'" ] }

Use dissect or grok to extract the timestamp from the message and then use a date filter to parse that.

[shani]

Hi Sir,

I'm unable to understand what's missing here. Dissect or Grok gets failed when I use to parse it. Like dissect is getting failed here

input {
file {
path => "/etc/logstash/files/time.log"
start_position => beginning
sincedb_path => "/dev/null"
}
}

filter {
dissect {
  mapping => {
    "message" => "%{log_time} TYPE=%{[event][type]} USER=%{[user][name]} COMP=%{[most][hostname]} SORC=%{[event][module]} CATG=%{[event][category]} EVID=%{[event][id]} MESG=%{[custom][message]}"
  }
}
date {
match => ["log_time", "d/M/YYYY HH:mm a" ]
target => "[event][created]"
}  
}

output {
stdout { codec => rubydebug }
}

Input files contains the logs
3/2/2020 10:16 AM TYPE=Information USER= COMP=ABC.xyz.local SORC=Software Protection Platform Service CATG=(0) EVID=1003 MESG=The Software Protection service has completed licensing status check. Application Id=0ff1ce15-a989-479d-af46-f275c6370663 Licensing Status= 1: 149dbce7-a48e-44db-8364-a53386cd4580, 1, 1 [(0 [0x00000000, 1, 0], [(?)( 1 0x00000000)(?)( 2 0x00000000 3 0 msft:rm/algorithm/hwid/4.0 0x00000000 0)(?)( 9 0x00000000 46 34702)(?)(?)])(1 )(2 )(3 )]

[Badger]

@shani

dissect { mapping => { "message" => "%{log_time} TYPE=%{[event][type]} ...

That is not going to match. It says that [message] should contain characters that are not space (the delimiter), followed by a space, followed by the literal string TYPE=. That's not what you have. Try

dissect { mapping => { "message" => "%{log_time} %{+log_time} %{+log_time} TYPE=%{[event][type]}