Date type fields not being created though date format conversion works

Elastic Stack Logstash
1

Hi, I am collecting http access logs and wanting to place some date headers into date type fields. I've tried a couple of things, shown below, but while the date format is converted properly, the resulting field is still of type keyword, rather than date.

To be clear, I'm receiving RFC 1123 formatted dates and they are being converted to the Elasticsearch date format (right?):

Wed, 25 Nov 2020 19:24:27 GMT ---> 2020-11-25T19:24:27.000Z

But the resulting field is still a string type.

date{
  match => ["[respHdr][date]", "EEE, dd MMM yyyy HH:mm:ss zzz"]
  target => "[http][response][date]"
  remove_field => [ "[respHdr][date]" ]
}

date{
  match => ["[http][response][expires]", "EEE, dd MMM yyyy HH:mm:ss zzz"]
  target => "[http][response][expires]"
}

I do see that placing a date into @timestamp (the default target) works fine:

   date{
    match => ["start", "UNIX"]
  }

I have deleted the index and started from fresh each time, but am seeing the same behavior. I'm using the ecs-logstash index template and ELK 7.10.

I see that date_detection is false by default in the ECS template, but not sure if that's the issue.

I'd appreciate any pointers on getting this to work and am happy to provide any further needed information.

Thanks,

--Chris

2

I believe it is. If you index template does not map the field as a date and date_detection is disabled I cannot think of a way the field would get mapped as a date.

3

Thanks! So I guess it would be a question for the elasticsearch forum about whether changing this to true would be a bad thing, or if there's anything else I can do...

Thanks again,

--Chris

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.