Logstash Date Filter not working on Kibana

[userR]

date {
match => ["logdate", "YYYY.MM.dd HH:mm:ss,SSS"]
target => "logdate"
}

Above is my date filter configuration on my logstash config file. On kibana, when I create an index pattern, the only date option that shows up is @timestamp but not logdate.

[aaron-nimocks]

What is the mapping of logdate? Have a sample of the data?

[userR]

There is a logdate field which I believe parses correctly. Here is an example of the log time format: 2020-08-13 15:57:14,242

Here is my config file below:
    input {
      beats {
        port => "5044"
        host => "0.0.0.0"
        ssl => false
      }
    }
    filter {
      grok {
        match => [
          #*grok patterns*
        ]
      }
      date {
        match => ["logdate", "YYYY.MM.dd HH:mm:ss,SSS"]
        target => "logdate"
      }
    }
    output {
      elasticsearch {
        hosts => ["*hosts*"]
        index => "logdata-%{+YYYY.MM.dd}"
      }
      stdout{}
    }

[Badger]

What is the mapping for the field in elasticsearch? If is mapped as a string, then even if the date filter successfully parses it, elasticsearch will convert it from a Logstash::Timestamp to a string as it gets indexed.

[userR]

"logdate" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }

[Badger]

OK, so it is mapped as text. You will need to start over with a new index, if the date filter is working it will get mapped as a date/time. You could use an index template to force this mapping but probably will not need to.

[userR]

This is the 2nd index I have tried to use the date filter with, and in both cases logdate does not show up for time. How would I use the index template? I am not familiar with that.

[Badger]

It is documented here.

[elasticforme (Sachin)]

@userR

date {
        match => ["logdate", "YYYY.MM.dd HH:mm:ss,SSS"]
        target => "logdate"
      }

and your date is with -

2020-08-13 15:57:14,242

you need to change to YYYY-MM-dd

[userR]

Yep that's it, sorry for the trouble!