"_dateparsefailure" due to Timestamp Cisco IOS

Elastic Stack Logstash
1

I used the conf file from here and the pattern below in a pattern file referenced by the conf file.

.conf grok section

grok {
  # There are a couple of custom patterns associated with this filter.
  patterns_dir => [ "/etc/logstash/conf.d/patterns/ios" ]

  match => [
    # IOS
    "message", "%{SYSLOG5424PRI}(%{NUMBER:log_sequence})?:( %{HOSTNAME:hostname}:)? (%{INT:cisco_seq_num}: )?.?%{CISCOTIMESTAMPTZ:log_date}: %%{CISCO_REASON:facility}-%{INT:severity_level}-%{CISCO_REASON:facility_mnemonic}: ( )?%{GREEDYDATA:message}",
    "message", "%{SYSLOG5424PRI}(%{NUMBER:log_sequence})?:( %{HOSTNAME:hostname}:)? (%{INT:cisco_seq_num}: )?.?%{CISCOTIMESTAMPTZ:log_date}: %%{CISCO_REASON:facility}-%{CISCO_REASON:facility_sub}-%{INT:severity_level}-%{CISCO_REASON:facility_mnemonic}: ( )?%{GREEDYDATA:message}"
  ]

  overwrite => [ "message" ]

  add_tag => [ "ciscoios" ]

  remove_field => [ "syslog5424_pri", "@version" ]
}
}

pattern file /etc/logstash/conf.d/patterns/ios/patterns-ios:

CISCOTIMESTAMPTZ %{CISCOTIMESTAMP}( %{TZ})?

the %{TZ} pattern is in the default grok patterns for logstash as:

TZ (?:[APMCE][SD]T|UTC)

I'm receiving "_dateparsefailure" ONLY for logs that specify a timestamp OTHER THAN "EST" e.g. CST|PST.

If I change the clock on the Cisco device to use EST rather than PST|CST the "_dateparsefailure" stops.

2

You have to share how your input log looks, then someone will be able to help you

3

Ah of course, thanks!

The parse fail happens for only the PST and CST timestamped logs.

<189>261731: HOSTNAME1: Nov 27 08:00:00.838 PST: %SYS-5-CONFIG_I: Configured from console by username on line# (XX.XX.XX.XX)
<189>14022: HOSTNAME2: Nov 27 09:58:01.178 CST: %SYS-5-CONFIG_I: Configured from console by username on line# (XX.XX.XX.XX)
<189>183040: HOSTNAME3: 183040: Nov 27 10:45:10.946 EST: %SYS-5-CONFIG_I: Configured from console by username on line# (XX.XX.XX.XX)

Parse as such below.

{
                 "host" => "XX.XX.XX.XX",
          "fingerprint" => "8b13ea19088714a1f29feb6201397e76436ed2e8",
             "hostname" => "HOSTNAME1",
             "facility" => "SYS",
         "log_sequence" => "261735",
             "log_date" => "Nov 27 08:02:21.024 PST",
        "facility_full" => "Operating system",
           "@timestamp" => 2019-11-27T16:02:20.444Z,
                 "tags" => [
        [0] "ciscoios",
        [1] "_dateparsefailure"
    ],
                 "type" => "syslog-cisco",
    "facility_mnemonic" => "CONFIG_I",
              "message" => "Configured from console by username on line# (XX.XX.XX.XX)",
       "severity_level" => "5 - Notification"
}

and

{
              "message" => "Configured from console by username on line# (XX.XX.XX.XX)",
           "@timestamp" => 2019-11-27T15:57:06.890Z,
        "facility_full" => "Operating system",
             "facility" => "SYS",
                 "host" => "XX.XX.XX.XX",
                 "tags" => [
        [0] "ciscoios",
        [1] "_dateparsefailure"
    ],
    "facility_mnemonic" => "CONFIG_I",
       "severity_level" => "5 - Notification",
             "hostname" => "HOSTNAME2",
                 "type" => "syslog-cisco",
          "fingerprint" => "78c95a9ccebe1fa19d14241c251ac7d28381b2eb",
         "log_sequence" => "14021",
             "log_date" => "Nov 27 09:57:05.879 CST"
}

and

{
              "message" => "Configured from console by username on line# (XX.XX.XX.XX)",
           "@timestamp" => 2019-11-27T15:54:03.939Z,
        "facility_full" => "Operating system",
        "cisco_seq_num" => "183041",
             "facility" => "SYS",
                 "host" => "XX.XX.XX.XX",
                 "tags" => [
        [0] "ciscoios"
    ],
    "facility_mnemonic" => "CONFIG_I",
       "severity_level" => "5 - Notification",
             "hostname" => "HOSTNAME3",
                 "type" => "syslog-cisco",
          "fingerprint" => "8f01d06ce0e47565930c10bd7c468bf7c0021c8e",
         "log_sequence" => "183041",
             "log_date" => "Nov 27 10:54:03.939 EST"
4

Nov 27 08:00:00.838 PST(or any other EST)
%{CISCOTIMESTAMP:log_date} %{GREEDYDATA:timestamp}

Try the above in grok debugger , and mutate your whole filter accordingly.Go step by step while building grok.

5

OK, so the grok works just fine, the issue is that you are getting a _dateparsefailure when you pass [log_date] to a date filter. That happens because PST and CST are ambiguous. There is no way for the date filter to know what PST refers to. You can use mutate+gsub to disambiguate it by replacing PST with US/Pacific, or PST8PDT, or Pacific/Pitcairn depending on which one you are using.

6

Yea I just noticed that in the logstash docs referencing the joda timzones. I'll try your solution

Thanks!

7

I used this below and it worked perfectly thanks!

mutate{
                gsub => [
               "log_date", "EST|EDT", "EST5EDT",
               "log_date", "CST|CDT", "CST6CDT",
               "log_date", "MST|MDT", "MST7MDT",
               "log_date", "PST|PDT", "PST8PDT"
        ]
        }
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.