Multiple Date formats in one file

[Jaxon_Kochel]

Hello,

I'm having an issue with date parsing multiple dates in one file. is there anyway to do multiple message matches / GROKS depending on what the log line is?

For instance:

USING this GROK
match => { "message" => "%{DATE:date} -? ?%{TIME:time} %{LOGLEVEL:severity} ?[%{NOTSPACE:package}]\ ?%{GREEDYDATA:message}"
}

So that would match a log line of this type

06/09/2016 - 16:22:42.178 PM [INFO ] [com.arrowstream.shipmentstatus.ShipmentStatusReceiveFM] Scanning directory \ASPD-EBIFS01\Extol\Inbound...

But...
In that same log file there are other lines written with different date formats. So I know I would need a different grok. is it possible to say her if the line matches this grok then do this?

sample other lines in same log file

2016-04-21 13:46:23,946 INFO [org.apache.shiro.realm.AuthorizingRealm] No cache or cacheManager properties have been set. Authorization cache cannot be obtained.

06/10/2016 - 11:42:41.704 AM [INFO ] [com.arrowstream.shipmentstatus.ShipmentStatusReceiveFM] Scanning directory \ASPD-EBIFS01\Extol\Inbound...

As you can see the data is ultimately written in three different ways:
year-day-month HH:mm:ss,SSS
year-month-day HH:mm:ss,SSS
day/month-year HH:mm:ss,SSS

So it there anyway I can account for this?

[magnusbaeck (Magnus Bäck)]

Both grok filters and date filters can list multiple expressions that will be tried in order until there's a match.

[Jaxon_Kochel]

So I can do something like this correct?

grok{
match => {
"message" => "%{DATESTAMP:timestamp} %{LOGLEVEL:severity} ?[%{NOTSPACE:package}] ?%{GREEDYDATA:message}"
}
match => {
"message" => "%{DATE:date} - %{TIME:time} %{WORD:AM/PM}\s*?[%{LOGLEVEL:severity}\s*] [%{NOTSPACE:package}] %{GREEDYDATA:message}"
}
}

---

So then how would I create a new field based upon which grok field gets selected?

as in for the first one I would need
mutate{
add_field => {"timestampTest" => "%{timestamp}"}
}

and the second I would use

mutate{
add_field => {"timestampTest" => "%{date} %{time}"}
}

[Jaxon_Kochel]

I have discovered the answer to both my questions now I really appreciate all the work you do on here Magnus.

Yes to part one

and then for part two i can do

if [date] {} to check if the date field exists to add a unique timestamp.