Failed to parse [date]

Elastic Stack Logstash
1

I hope someone can guide me as for date parsing.

Logstash config has below date format.

date {
match => [ "@date", "yyyy-MM-dd HH:mm:ss Z" ]
}

But, I am getting below error...
"status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [date]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "2018-05-04 03:55:30 +0000" is malformed at " 03:55:30 +0000""}}}}}

When I run manually, I received the output as below.

              "date" => "2018-05-04 03:55:30 +0000",
"bytes_written_to_client" => "4933",
                  "srcip" => "x.x.x.x",
                "dstport" => "443",
"bytes_written_to_server" => "3303",
                    "pid" => "18895",
                   "type" => "t_nettraffic",
               "hostname" => "FIREWALLONE",
                   "host" => "10.10.10.10",
                "devname" => "FIREWALL1",
                  "dstip" => "x.x.x.x",
                  "event" => "session end",
              "rule_name" => "TEST RULE",
                    "pri" => "p_major",
               "priority" => "45",
                "dst_geo" => "XX",
                "srczone" => "VL_SOURCE",
             "start_time" => "2018-05-04 03:54:25 +0000",
             "@timestamp" => 2018-05-04T03:55:30.167Z,
            "application" => "TCP 443",
                  "proto" => "6",
                "srcport" => "57878",
                "dstzone" => "VLEXT",
                  "logid" => "0",
                    "cmd" => "httpp"
2

It looks like your field is named date and not @date, which is what you have specified in the date filter.

3

Sorry, my typo.
Correct one don't have @ sign.

date {
match => [ "date", "yyyy-MM-dd HH:mm:ss Z" ]
}

4

This works for me:

input {
  generator {
    lines => ['2018-05-04 03:55:30 +0000']
    count => 1
  } 
} 

filter {
    date {
        match => ["message", "yyyy-MM-dd HH:mm:ss Z"]
    }
}

output {
  stdout { codec => rubydebug }
}
1 Like
5

Thanks, but it is still getting error as below. But, when I execute logstash, I am still getting like your output.
Elasticsearch is not showing log from this firewall.

[2018-05-04T14:11:05,745][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2018.05.04", :_type=>"t_nettraffic", :_routing=>nil}, 2018-05-04T06:11:05.000Z x.x.x.x %{message}], :response=>{"index"=>{"_index"=>"logstash-2018.05.04", "_type"=>"t_nettraffic", "_id"=>"AsdfwesxxERDSFWExxrfwere", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [date]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "2018-05-04 06:11:05 +0000" is malformed at " 06:11:05 +0000""}}}}}

6

@Christian_Dahlqvist ...Thanks.
Finally solved it.
I just added remove_field => ["date"].

Here is completed one.

date {
match => [ "date", "yyyy-MM-dd HH:mm:ss Z" ]
remove_field => ["date"]
}
}

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.