Custom grok filter

Elastic Stack Logstash
1

Hello, I'm new to all this so bear with me. My company has asked me to figure out elk stack and how to make it work for their logs. I have everything set up and working but I need to be able to parse the logs in kibana a certain way. I am guessing that I would use grok filter in logstash.conf for that right?
I need to filter and see the message like so:
node:
domain:
group:
profile:
timeofreceipt:
unknown:
format:
unknown:
timein:
timeenq:
timedeq:
timesent:
cat:

Does anyone know how I can do that or where I can learn how to?

Thanks

2

Do you have an example of one of your log files?

3

Hi Aqua,

They said I cant post all of the message example due to being proprietary but this is how it looks: node.domain.group.profile.timeofreceipt.unknown.format.unknown.timein.timeenq.timedeq.timesent.cat

I replaced the each part of the message with what category it is.

Here is part of the message i can post though: This what each time number means,
timein: 2021:07:27:15:48:16:579463060
timeenq: 2021:07:27:15:48:46:588113175
timedeq: 2021:07:27:15:48:16:588134625
timesent: 2021:07:27:15:48:16:5881858865

And this is the actual message for what the times look like: 2021:07:27:15:48:16:579463060,2021:07:27:15:48:46:588113175,2021:07:27:15:48:16:588134625,2021:07:27:15:48:16:5881858865

So basically, I need to be able to somehow parse the message in kibana so they can see each category. I hope this makes some sense to you. Im only 2 days into elk stack in general so just figuring out the basics atm.

4

Our Splunk lady helped me figure it out so I am posting the answer in case anyone else ends up needing to do this.
For grok filter I added this:
grok {
match => { "message" => "(?\w+).(?\w+).(?\w+).(?\w+)|(?\d+.\d+)|(?\d+)|(?\S+)|(?\d+),(?\S+),(?\S+),(?\S+),(?\S+),(?\d)" }
}
}

Now in Kibana, I can see the fields I need and the message is parsing correctly.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.