Hi @Badger,
I think i've made some limited progress based on what you've said and following this post
My Grok filter / date filters now look like this:
grok {
match => {
"message" => "(?<serverid>([a-z]{3}[0-9]{2}[a-z]{2}[0-9]{2}){0,1})\s+(?<username>([A-Za-z0-9.@\-]*){0,1})\s+(%{TIMESTAMP_ISO8601:logdate})?\s+(%{LOGLEVEL:loglevel})\s+(?<classname>([^\s]+){0,1})\s+%{GREEDYDATA:body}"
}
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
tag_on_failure => [ "_not_interested" ]
}
date {
match => [ "logdate", "YYYY-MM-dd HH:mm:ss,SSS" ]
target => "logdate"
}
I was hoping that, following this change, when i created a new index pattern, i'd have the option of using "logdate" as the time field, but it isn't showing in the list and, if i proceed to create the index pattern on @timestamp then logdate still shows as a String in the field list on the next page.
I can see its made a change to the output in Discover view now as the Logdate is now showing slightly differently to the Original Input:
Before
2020-12-14 12:18:51,397
After
2020-12-14T12:18:51.397Z
Any further suggestions?
Many thanks,
Ade.