Issue parsing non-standard timestamp (Cisco router log)

[AJ_NOURI]

I am trying to parse logs from Cisco router with the following format:

000022: Jan 02 15:42:11.048:%LINK-5-CHANGED: Interface Ethernet0, changed state to administratively down

Notice the particular timestamp:

Jan 02 15:42:11.048

for which there is no prefedined grok pattern that contains the semantic {MONTH} {MONTHDAY} and something like TIME with milliseconds

I am using the following grok match:

filter {
 grok {
 match => {"message" => '%{INT:local_seq_num:int}: %{INT:src_seq_num:int}: (?<IOSVTIMESTAMP>): \%%{WORD:facility}\-%{WORD:severity}\-%{WORD:mnemonic}: %{GREEDYDATA:description}'} 
 }
}

The result is as expected except the timestamp, I don't know what to do with it:

How can I create a grok pattern that correctly parse the timestamp as such so it can be used in elasticsearch?

[Badger]

Using dissect instead of grok...

dissect { mapping => { "message" => "%{}: %{ts}:%{+ts}:%{+ts}:%{}" } }
date { match => [ "ts", "MMM dd HH:mm:ss.SSS" ] remove_field => [ "ts" ] }

You will probably need a timezone option on that date filter.

[AJ_NOURI]

Hi @Badger, thanks for your reply.

I have learnt about dissect and the expression makes sens to me.
I have some questions:

• Why you remove the "ts" field at the end of date? Then where the parsed date is stored? In @timestamp?
• How to integrate it to the previous work? Should I remove the previous grok filter and redo everything in dissect? Or use them both with dissect for the time?

[Badger]

The remove_field will happen if date successfully matches ts to that pattern. If it does not match it will not be removed. The date will be stored in @timestamp, and there is no need to keep ts if the parsed value has been stored.

It would make sense to completely replace grok with dissect, but then you need to add the rest of the fields to the dissect. The image you posted does not match the text of your post. For the image...

"<%{level}>:%{local_seq_num}: %{src_seq_num}:  *%{ts} %{+ts} %{+ts} %{+ts} %{+ts}: %%{facility}-%{severity}-%{mnemonic}: %{description}"

[AJ_NOURI]

Thanks a lot @Badger, now it works fine.

I ended up with this formula:

dissect {
        mapping => { "message" => "<%{local_seq_num}>%{src_seq_num}: *%{ts} %{+ts} %{+ts}:%{+ts}:%{+ts}.%{+ts}: %%{facility}-%{severity}-%{mnemonic}: %{description}"}
    }
    date {
        match => ["ts", "MMM dd HH:mm:ss.SSS" ] remove_field => [ "ts" ]
    }