Date filter to yyyy/MM/dd HH:mm:ss format

Elastic Stack Logstash
1

Hello,

I´m trying to parse a date field called "ReceiveTime" which has the following format:
ReceiveTime=2020/03/03 16:47:56

i´m using the next filter in Logstash:
date {
match => [ "ReceiveTime", "yyyy/MM/dd HH:mm:ss"]
timezone => "America/Chicago"
}

The thing is I got a _dateparsefailure, it´s not working.
Anybody knows how to parse this kind of date using date filter? I would like to ingest the logs with this timestamp field, called ReceiveTime.
Thank you ver much in advance.

Regards

2

I would expect that date filter to correctly parse that string. If you use

output { stdout { codec => rubydebug } }

then what do the ReceiveTime and @timestamp fields look like in the output.

3

@Badger this is part of the ouput using the rubydebug:

     "SessionID" => "0009",
                 "Flags" => "0x4000",
              "sequence" => "6632513",
          "totalPackets" => "1",
            "totalBytes" => "66",
"DeviceGroupHierarchyL2" => "0",
          "SerialNumber" => "0001",
             "StartTime" => "2020/03/03",
       "DestinationZone" => "web",
            "@timestamp" => 2020-03-23T10:26:26.656Z,
               "srcPort" => "port",
       "ParentSessionID" => "0",
                 "proto" => "tcp",
      "IngressInterface" => "ethernet1",
           "ActionFlags" => "0x0",
              "@version" => "1",
          "ActionSource" => "from-policy",
               "dstPort" => "443",
      "SessionEndReason" => "policy-deny",
              "RuleName" => "interzone-default",
           "URLCategory" => "any",
            "SyslogInfo" => "deny",
                   "dst" => "ip",
           "ReceiveTime" => "2020/03/03",
                  "tags" => [ [0] "_dataparsefailure" ],

Any idea? maybe problem is in HH:mm:ss, because that part seems to be empty.
Btw, as I told before the value of this field is, for example:
ReceiveTime=2020/03/03 16:47:56

Thank you very much

4

Well it is not, is it. It is quite clearly

"ReceiveTime" => "2020/03/03",

with no time part. Maybe you need a second pattern to match events where the time is missing

match => [ "ReceiveTime", "yyyy/MM/dd HH:mm:ss", "yyyy/MM/dd" ]
5

When I see the source log I can see the ReceiveTime value, and it´s like I said before.
I think maybe the problem is kv is not filtering properly the value field, so only date part is parsed but not time part.
Could it be?

This is part of the source log, which includes the ReceiveTime field:
ReceiveTime=2020/03/03 16:47:56|SerialNumber=099123|cat=THREAT|Subtype=vulnerability|

Regards

6

How is your kv filter configured?

7

It´s solved. It was kv problem.

Thanks.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.