Elastic Search Timestamp format

[jakelaws]

Hi,

I am currently have NiFi ingesting data into our elastic search environment. We are trying to get elastic to recognize the timestamp format '2017-12-04 18:25:30.114000+00:00' but no matter what we have tried it fails and throws the error of illegal argument exception, invalid format. Please advise on how we can get Elastic to recognize this format.

Thank you.

[rjernst (Ryan Ernst)]

Can you show which format strings you have tried and what method you have used to setup the field with that format?

[jakelaws]

I tried pushing in the data and creating a new index and seeing if it recognized a time-based field, which it did not.

I also tried preformatting
PUT {index}
{
"mappings": {
"{typeName}": {
"properties": {
"{fieldName}" : {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss.SSSSSSZZ"
}
}
}
}
}

also tried format: strict_date_optional_time

[rjernst (Ryan Ernst)]

What do you mean by "it did not"? Did you get a specific error when creating the index?

[jakelaws]

No, when selecting "Index contains time-based events" and then trying to select a drop down of fields, it does not recognize any field is a time-based field. Although the data has time field with the time format specified above.

[rjernst (Ryan Ernst)]

You seem to be referring to a UI. Maybe Kibana? I would suggest asking this in the Kibana discuss forum since in this one we really only know the direct elasticsearch API.

[jakelaws]

Makes sense, I just figured it may be this one since the NiFi PutElasticSearch is rejecting the data insert. I will post in that discussion form, thank you for the guidance.