Date parse failure message strange behavior

Elastic Stack Logstash
1

Hello,

Using the following filter :

#log timestamp
filter {
date {
    match => [ "timestamp","UNIX" ]
    target => "log_timestamp"
    timezone => "Europe/Paris"
}
}

I have a [0] "_dateparsefailure", but target field (log_timestamp) is well created :

{
           "sentpkt" => "3",
              "date" => "2020-01-06",
       "srcintfrole" => "undefined",
        "policytype" => "policy",
    "severity_label" => "Notice",
           "service" => "tcp/44814",
             "dstip" => "1.2.3.4",
             "logid" => "0000000013",
             "proto" => "6",
          "duration" => "12",
         "logsource" => "4.5.6.7",
          "craction" => "262144",
         "sessionid" => "696656618",
          "@version" => "1",
            "logver" => "60",
           "srcintf" => "LAN_P1",
          "trandisp" => "snat",
          "srcgeoip" => {},
           "srcport" => "50026",
              "type" => "traffic",
             "srcip" => "192.168.1.1",
         "eventtime" => "1578300095",
           "transip" => "1.2.3.4",
        "@timestamp" => 2020-01-06T08:41:40.000Z,
          "priority" => 189,
          "severity" => 5,
           "devname" => "FGT",
                "tz" => "UTC+1",
              "time" => "09:41:35",
     "log_timestamp" => 2020-01-06T08:41:35.000Z,
           "rcvdpkt" => "2",
          "facility" => 23,
              "tags" => [
        [0] "_dateparsefailure",
        [1] "_geoip_lookup_failure"
    ],
            "action" => "timeout",
           "dstport" => "44814",
        "srccountry" => "Reserved",
           "crlevel" => "low",
       "dstintfrole" => "undefined",
            "appcat" => "unscanned",
             "level" => "notice",
           "dstintf" => "WAN_2",
           "subtype" => "forward",
           "crscore" => "5",
             "devid" => "FG123456",
          "rcvdbyte" => "176",
          "sentbyte" => "180",
    "facility_label" => "local7",
           "poluuid" => "d8ff503c-cdca-51e7-d709-e625957be00d",
                "vd" => "PRT",
         "transport" => "50026",
           "message" => "logver=60 timestamp=1578300095 tz=\"UTC+1\" devname=\"FGT\" devid=\"FG20\" vd=\"PORTAIL\" date=2020-01-06 time=09:41:35 logid=\"0000000013\" type=\"traffic\" subtype=\"forward\" level=\"notice\" eventtime=1578300095 srcip=1.2.3.4 srcport=50026 srcintf=\"LAN_1\" srcintfrole=\"undefined\" dstip=4.5.6.7 dstport=44814 dstintf=\"WAN_2\" dstintfrole=\"undefined\" poluuid=\"d8ff503c-cdca-51e7-d709-e625957be00d\" sessionid=696656618 proto=6 action=\"timeout\" policyid=63 policytype=\"policy\" service=\"tcp/44814\" dstcountry=\"France\" srccountry=\"Reserved\" trandisp=\"snat\" transip=4.5.6.7 transport=50026 duration=12 sentbyte=180 rcvdbyte=176 sentpkt=3 rcvdpkt=2 appcat=\"unscanned\" crscore=5 craction=262144 crlevel=\"low\"\n",
          "policyid" => "63",
        "dstcountry" => "France",
              "host" => "1.2.3.4",
         "timestamp" => "1578300095"
}

So I don't understand this message.

What is strange is I have other logs from a similar network device with the same filter and it works without any issue :

{
              "time" => "10:06:16.665",
     "log_timestamp" => 2020-01-06T09:06:16.000Z,
          "facility" => 23,
        "session_id" => "00696GKc016415-00696GKd016415",
    "severity_label" => "Informational",
            "log_id" => "0300016416",
         "logsource" => "1.2.3.4",
          "@version" => "1",
                "to" => "'undisclosed-recipients:'@webmail.bonanga.com",
               "pri" => "information",
         "device_id" => "FE-3KD3R15000024",
           "subject" => "RE: Update yolo - 15/10/2019_1",
            "nested" => "File name: image001.png scanned by Antivirus Scannerclean",
              "type" => "spam",
            "dst_ip" => "10.255.50.205",
    "facility_label" => "local7",
        "@timestamp" => 2020-01-06T09:06:16.000Z,
       "client_name" => "webmail.bonanga.com",
          "priority" => 190,
         "client_ip" => "4.5.6.7",
                "vd" => "root",
           "message" => "timestamp=1578301576 date=2020-01-06 time=10:06:16.665 devname=\"FE-AZERTY\" device_id=\"FE-AZERTY\" log_id=\"0300016416\" type=\"spam\" pri=\"information\" session_id=\"00696GKc016415-00696GKd016415\" client_name=\"webmail.bonanga.com\" client_ip=\"4.5.6.7\" dst_ip=\"10.255.50.205\" from=\"alex@piou.fr\" to=\"'undisclosed-recipients:'@webmail.bonanga.com\" subject=\"RE: Update yolo - 15/10/2019_1\"  nested=\"File name: image001.png scanned by Antivirus Scannerclean\" vd=\"root\"\n",
          "severity" => 6,
           "devname" => "FE-AZERTY",
              "host" => "192.168.160.5",
         "timestamp" => "1578301576",
              "from" => "alex@piou.fr"
}

Thanks for your feedback !

2

If the target field is getting created then the date filter is not failing and will not add the _dateparsefailure tag. That would suggest you have another date filter in your configuration. Try running with --config.debug --log.level debug --config.test_and_exit on the command line, and verify that you are using the configuration that you think you are using.

3

Indeed I had another date filter not in a dedicated global filter. After have fixed the conf, no more date parse failure message :slight_smile: Thanks a lot @Badger !

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.