How to Parse and Copy a Field

Elastic Stack Logstash
1

I have the TimeStamp field and I would like to extract the hour of the day so that I can look for out of hours user activity.

Jan 3 12:09:01

filter {
grok {
match => { "TimeStamp" => "%{WORD}\s%{NUMBER}\s%{NUMBER:HourOfDay}:%{NUMBER}:%{NUMBER}" }
}
}

At the top you can see an example of TimeStamp and below is how I would go about parsing it. I would like to however on parse the hour of the day into the new field HourOfDay and then keep the TimeStamp field as is.

Would this grok config work?

Cheers,

G

2

Would this grok config work?

Why not try it out? It looks okay but you'll probably want %{NUMBER:HourOfDay:int} to make the field an integer.

3

Hi @magnusbaeck I tried changing my config to the above and restarted Logstash, upon restarting my whole VM locked up and I had to restart the entire thing! Logstash didn't write to any of the logs (it's set to debug mode) and I had to just remove the grok pattern.

I'm unsure where to go from here.

Cheers,

G

4

I find it very hard to believe that that configuration change crashed the whole VM.

5

Hi there @magnusbaeck I think I found out why the VM crashed. Logsash log level was set to DEBUG, once I got back into the VM (as Logstash doesn't automatically start when the VM does) I found there to be around 15GB of log files created by Logstash. I will set the log level to normal and retry the grok.

Cheers,

G