Droping log when missing time entry

[iccMe (Ian)]

Hi,

I was wondering if anyone could advise the best way to drop a log when the actual time is missing from the logged event. Normally the majority of the logs look like:

[19/10/2020 00:00:01] servername - Processor Queue Ok 0 WMI (localhost:ProcessorQueueLength) 3949

however there is some entries that look like:

[19/10/2020] servername - Processor Queue Ok 0 WMI (localhost:ProcessorQueueLength) 3935

My grok pattern is still able to match and filter the message, but I get a _dateparsefailure tag added in Kibana due to the time being missing. How could I get logstash to drop any event that is missing the time? Below is the grok example I would use for the logs above, however the missing time is not unique to that type of event.

grok {
			match => { "message" => "%{SYSLOG5424SD:Time}%{SPACE}%{HOSTNAME:hostname} - (?<event>[^\t]*)%{SPACE}%{WORD:status}%{SPACE}%{NUMBER:reply}%{SPACE}%{GREEDYDATA:drop}" }
	}

thanks

Ian

[Badger]

If you want to drop events that have a _dateparsefailure tag then you could use

if "_dateparsefailure" in [tags] { drop {} }

[iccMe (Ian)]

Hi @Badger,

I should have said I tried that, but it seems to ignore that condition.