Specifying custom date fields in an Elasticsearch index template


(Al) #1

I've tried updating my index template to allow an epoch and ISO8601 date format directly at the mapping level in Elasticsearch, which should override the @timestamp field. So far I've had no luck. My template can be viewed here http://pastebin.com/1DCT6GfM.

Essentially, once I parse the json logs are parsed by Logstash, Elasticsearch should recognized those fields and automatically update the @timestamp field. Examples of potential log events include:

{"timestamp_epoch": "1432146610", "message": "This is a test message to test epoch timestamp events", "category": "testing", "country": "EU"}

or

{"timestamp_iso8601": "2015-05-20T14:30:20-05:00", "message": "This is a test message to test epoch timestamp events", "category": "testing", "country": "US"}

Please note that every other mapping in that template works fine except for timestamp_epoch and timestamp_iso8601. I can tell those two mappings aren't taking affect because the @timestamp isn't updating to the proper value.

You can see a few of the raw sample Elasticsearch entries here: http://pastebin.com/HmtmNNAb

The type for timestamp_iso8601 is updating to date, but it's not overriding the @timestamp field and timestamp_epoch set to string instead of date

I'd really appreciate if someone could clarify exactly what it is that i'm doing incorrectly.

Thanks for the help!


(system) #2