Hi, Good evening
I'm parsing json logs to elasticsearch using logstash json filter. I'm getting _dataparsefailure in Kibana dashboard and I couldn't see any error logs in logstash regarding this.
time format in logs : "2021-03-02T18:30:00.007Z"
Logstash input configuration is:
>
> filter {
> if "dev" in [tags] {
> json {
> source => "message"
> }
> mutate {
> rename => ["timestamp", "logtime" ]
> }
> date {
> match => [ "logtime", "YYYY-MM-dd HH:mm:ss.SSS", "YYYY-MM-dd HH:mm:ss.SS" ]
> target => "logtime"
> }
> }
> }
filebeat log output:
> {
> "log" => {
> "offset" => 586,
> "file" => {
> "path" => "/home/kund0831/json-dev.log"
> }
> },
> "host" => {
> "name" => "kund0831"
> },
> "@version" => "1",
> "logtime" => "2021-03-02T18:30:00.035Z",
> "message" => "hello",
> "@timestamp" => 2021-03-03T17:50:41.770Z,
> "agent" => {
> "name" => "kund0831",
> "ephemeral_id" => "e6db5177-2beb-4fc5-b227-e20203de9acb",
> "version" => "7.10.2",
> "id" => "a6d5bbd9-5b06-469e-9733-917cd833a230",
> "hostname" => "kund0831",
> "type" => "filebeat"
> },
> "tags" => [
> [0] "dev",
> [1] "beats_input_codec_plain_applied",
> [2] "_dateparsefailure"
> ],
> "ecs" => {
> "version" => "1.6.0"
> },
> "input" => {
> "type" => "log"
> },
> "level" => "info",
> "label" => "Multitenancy"
> }
Can someone help on this why I'm getting _dateparsefailure in tags in Kibana?
How can I fix this?
Thanks