Help with index template format for date with 6 millis

Elastic Stack Elasticsearch
1

I have a time field that has dates like: 2018-09-21T08:10:22.441223-0400 and I tried to create my template with the following:

"format" : "YYYY-MM-dd'T'HH:mm:ss.SSSSSS-Z"
"format" : "YYYY-MM-ddTHH:mm:ss.SSSSSS-Z"
"format" : "YYYY-MM-dd'T'HH:mm:ss.SSSSSSZ"

All these fail with the reason of "failed to parse" caused by "illegal_argument_exception" reason "Invalid format: "2018-09-21T08:10:22.441223-0400" is malformed at "T08:10:22.441223-0400"

Suggestions?

EDIT:

Some additional info: Maybe I'm missing something, but the timestamp is in the same format and for that I'm simply using the logstash config of:

date {
match => [ "datefield" , "ISO8601" ]
}

is there a template equivalent? I have not been able to find at http://www.joda.org/joda-time/key_format.html or at http://www.joda.org/joda-time/apidocs/org/joda/time/format/ISODateTimeFormat.html

2

I think your pattern needs to use a lowercase y for the year. The following should work: yyyy-MM-dd'T'HH:mm:ss.SSSSSSZ.

Curious to why need to specify the pattern at all ? Is the default loosing some of the millisecond precision ?

3

None of the millisecond patterns showed a millisecond pattern to six characters...

https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html

I am going to try the lower case...

4

OK, I was dumping logs into a test index and had set the template to not define that field. I just went to make that change and I noticed that ES analyzed that field and decided it was a date.

On the other hand, your suggestion worked! Thanks.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.