Give events without timestamp the timestamp from the last event

i'm currently trying to give events from the initialization dialog (they have no timestamp) the timestamp from the next possible event which has a timestamp. the list is upside down which means the latest entry is at the top. The interesting part (logs with and without timestamp) look somehow like that:

 66 4  0 days 00:00:09  2015/12/28 10:19:59 Ring redundancy role HRP Client.
 67 4  0 days 00:00:08  2015/12/28 10:19:58 Cold start performed
 68 3  0 days 00:00:02   Port Monitoring Barrier disabled.
 69 3  0 days 00:00:02   Port Mirroring disabled.
 70 3  0 days 00:03:17   Link down on port 1
 71 3  0 days 00:00:23   Link up on port 1

I can distinguish the messages by simple doing:

 if  [message] =~ /^[ , 0-9]+\t\d+\t \d+ days \d\d:\d\d:\d\d \t[\d]+/\d\d\/\d\d/{
            grok {...}
            date {...}
}elseif [message] =~ /^[ , 0-9]+\t\d+\t \d+ days \d\d:\d\d:\d\d \t [\w]+/{
            grok {...}
            date {???}

The problem with that is obviously that the events without a timestamp get data dateparsefaliure.
That leads the my question, is there a possibility to take the last known timestamp (in this case 2015/12/28 10:19:58) and write it to all the following events down below?
I found something called aggregate filter but im not sure if i can use thatin my case, and if i can, how?

Thanks in advance!

You could do it with aggregate using something like this.

  mutate { add_field => { "static" => "1" } }
  if [message] =~ /2015/ {
    aggregate {
      task_id => "%{static}"
      code => "map['something'] = event.get('message')"
  } else {
    aggregate {
      task_id => "%{static}"
      code => "event.set('something', map['something'])"

Or you could do it using a ruby class variable. Use one of these in the first block and the other in the second.

    ruby { code => '@@t = event.get("timestamp")' }
    ruby { code => 'event.set("timestamp", @@t)' }
1 Like

Hey Badger,
i tried your suggestion with the ruby class variable because it was exactly what i was looking for.
It works great that way!
Thank you very much for your time :slight_smile:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.