Logstash Date parsing error from Kinesis

I have picked up a Logstash server to look after and I can see some incoming events from a Kinesis input that is failing to be indexed:

 Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"fargate-container-development-2020.01.16", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x7677b8d7>], :response=>{"index"=>{"_index"=>"fargate-container-development-2020.01.16", "_type"=>"doc", "_id"=>"1234567890", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [time]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"For input string: \"2020-01-16T11:29:38.157Z\""}}}}}

I thought that should be a simple fix and tried this:

filter {
  if "kinesis" in [tags] and "fargatecontainerlogs" in [tags] {
    date {
      match => [ "time", "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", "ISO8601" ]
      target => "@timestamp"
      remove_field => [ "time" ]
    }

    json {
      source => "message"
      remove_field => [ "message" ]
    }
  }
}

Any idea where I went wrong?

So I tried changing the order of the json parse and the date parse and it started working.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.