Failed to parse timestamp exception, (only) when using a new Elasticsearch index name

Hi,

I am using Elasticsearch, Kibana and Metricbeat 6.1.1. I would like to use a different index name, for Metricbeat, than the default one. But I have problems as shortly follow:

  1. When I start Metricbeat, with default preferences (index name, pattern etc.), I am able to record the data in Elasticsearch and inspect in Kibana without a problem. (the metricbeat index gets created correctly in Elasticsearch)
  2. Then, in the metricbeat.yml file, I simply changed the Elasticsearch output.elasticsearch.index to something custom, along with the indicated setup.template.name and setup.template.pattern accordingly:

output.elasticsearch.index: "my-index-name-%{[beat.version]}-%{+yyyy.MM.dd}"
setup.template.name: "my-index-name"
setup.template.pattern: "my-index-name-*"

  1. In this case, when Metricbeat starts up, I get the following exception:

failed to execute bulk item (index) BulkShardRequest [[my-index-name][0]] containing [index....] ...
... org.elasticsearch.index.mapper.MapperParsingException: failed to parse [@timestamp] ...
... Caused by: java.lang.IllegalArgumentException: Invalid format: "2018-04-02T07:24:39.162Z"...

  1. If I comment-out the custom output.elasticsearch.index setting, then all is fine again and the default metricbeat index is created without problem.

I do not understand why, without modifying any field mappings or other settings, it does not work for
my custom metricbeat index, yet works if I comment the setting out.

I have searched the reference(s) and forum posts for a possible answer. But I fail to understand what I am missing (or doing wrong).
I apologize in advance if the question already has an answer which I failed to find.

Thank you in advance!
Andrei

Hi,

I managed to find the problem, after searching and comparing type mapping specifications in my application. (I am not that experienced with the ELK stack yet.)

Unfortunately, it appears I had (have) an index template in code, that specified a different @timestamp format. The template was specified something like

{ "template": "my-*",....
"mappings": ... "properties": "@timestamp": { "type": "date", "format": "epoch_millis"}
.. }

and this caused the trouble with my-index-name index.

Anyway, hope this answer might help other beginners as myself...

Best regards,
Andrei.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.