Invalid date format for ingest timestamp

Elastic Stack Elasticsearch
1

I need some help with pipeline (set ingest.timestamp)

{
"set": {
  "field": "meta.timestamp",
  "value": "{{_ingest.timestamp}}"
}
}

I've defined this pipeline, and mapping for this field is

    type: date
    format: "yyyy-MM-dd'T'HH:mm:ss.SSSSSSZZ"

However when I try to reindex documents I got error

failed to parse field [meta.timestamp] of type [date] in document with id
"reason": "Invalid format: "2020-04-16T08:00:14Z" is malformed at "Z""

I am using ES 6.8.
Any clues? How its possible that {{_ingest.timestamp}} has not valid format ?

Similar when I do create lot of documents in short period of time, I got similar error (happen very rarely )

I've test it in Java

    public static void main(String[] args) {
        for (int i = 0; i < 999999; i++) {
            System.out.println(ZonedDateTime.now(ZoneOffset.UTC));
        }
    }

and it seems that Java 'truncate' output

2020-04-16T08:23:27Z
2020-04-16T08:23:27Z
2020-04-16T08:23:27.001Z
2020-04-16T08:23:27.001Z
2020-04-16T08:23:27.001Z

How is correct way to use ingest.timestamp, if its format is not consistent ?

2

Same on ES 7.6.2

3

Any updates?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.