Conditionals and Dates

(Jack ELK West) #1

Issue: I have a lot of different logs coming from 1 source and separate date formats.

Hey everyone I have a conditional logstash file that takes a single input and runs multiple conditionals to determine which log file it is and how to parse it. My output is then to put them into Elasticsearch. I have roughly 40 logs i'm parsing therefor the date's of each log are different and different standards. All logs come from a single input with a header. I do an initial GROK to parse the header and put the rest of the data in a separate Log_message field. At this point before the sort to parse each field in the log the date is embedded in Log_message.

Question1: What is the best way to pull the date out of the Log_message and set it as the timestamp field in elastic and kibana?

(Mark Walkom) #2

I want to say a date filter, but given your complexity I don't think that is what you want?

(Magnus B├Ąck) #3

Sounds like exactly what the grok and date filters are for. They both support trying multiple patterns and quitting on the first match they get.

(system) #4