Unusual timestamp data that needs fixing

Elastic Stack Elasticsearch
1

Hello all,

Elastic 7.9.2
Kibana 7.9.2

I have an index that has somehow ended up with a date 20000 years in the future.

I thought at first that the @timestamp field had simply been misinterpreted as text by elastic, which was causing shard failure warnings in kibana, and so I created a new index-

POST _reindex?wait_for_completion=true    
{
  "source": {
    "index": "openstack-22020.12.16"
  },
  "dest": {
    "index": "openstack-22020.12.16-new"
  }
}

Deleted the old index-

DELETE openstack-22020.12.16

Recreated the old index with a date/time format defined-

PUT openstack-22020.12.16
{
  "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date",
        },

(truncated for brevity)
However when I tried to reindex the openstack-22020.12.16-new back to my newly created openstack-22020.12.16, I get the following error.

{
  "took" : 2,
  "timed_out" : false,
  "total" : 2,
  "updated" : 0,
  "created" : 0,
  "deleted" : 0,
  "batches" : 1,
  "version_conflicts" : 0,
  "noops" : 0,
  "retries" : {
    "bulk" : 0,
    "search" : 0
  },
  "throttled_millis" : 0,
  "requests_per_second" : -1.0,
  "throttled_until_millis" : 0,
  "failures" : [
    {
      "index" : "openstack-22020.12.16",
      "type" : "_doc",
      "id" : "ku0SaXYB7PbJS_CzFndk",
      "cause" : {
        "type" : "mapper_parsing_exception",
        "reason" : "failed to parse field [@timestamp] of type [date] in document with id 'ku0SaXYB7PbJS_CzFndk'. Preview of field's value: '22020-12-16T12:01:48.000000000+11:00'",
        "caused_by" : {
          "type" : "illegal_argument_exception",
          "reason" : "failed to parse date field [22020-12-16T12:01:48.000000000+11:00] with format [ordinal_date_time_no_millis]",
          "caused_by" : {
            "type" : "date_time_parse_exception",
            "reason" : "Failed to parse with all enclosed parsers"
          }
        }
      },
      "status" : 400
    },
    {
      "index" : "openstack-22020.12.16",
      "type" : "_doc",
      "id" : "IO8TaXYB7PbJS_CzFcL1",
      "cause" : {
        "type" : "mapper_parsing_exception",
        "reason" : "failed to parse field [@timestamp] of type [date] in document with id 'IO8TaXYB7PbJS_CzFcL1'. Preview of field's value: '22020-12-16T12:03:09.000000000+11:00'",
        "caused_by" : {
          "type" : "illegal_argument_exception",
          "reason" : "failed to parse date field [22020-12-16T12:03:09.000000000+11:00] with format [ordinal_date_time_no_millis]",
          "caused_by" : {
            "type" : "date_time_parse_exception",
            "reason" : "Failed to parse with all enclosed parsers"
          }
        }
      },
      "status" : 400
    }
  ]
}

A closer inspection of the date and the index name shows that somehow, this has come in with the year set to 22020, and I'm not sure how to get the year back to a sane number.
I had a look at this answer here-

However I was not able to successfully update the date using the script examples there.

I created a new "test" version of the index and made it writeable-

PUT openstack-22020.12.16-test/_settings
{
  "index": {
    "blocks.write": false
  }
}

Then tried

POST openstack-22020.12.16-test/_update_by_query
{
    "query": {
    "match_all": {}
},
"script": {
"source": "ctx._source['@timestamp'] = OffsetDateTime.parse(ctx._source['@timestamp']).minusYears(20000).toString()"
}
}

I suspect this is because of how far out of range the time has become. The error is

{
  "error" : {
    "root_cause" : [
      {
        "type" : "script_exception",
        "reason" : "runtime error",
        "script_stack" : [
          "java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2051)",
          "java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1953)",
          "java.base/java.time.OffsetDateTime.parse(OffsetDateTime.java:403)",
          "java.base/java.time.OffsetDateTime.parse(OffsetDateTime.java:388)",
          "ctx._source['@timestamp'] = OffsetDateTime.parse(ctx._source['@timestamp']).minusYears(20000).toString()",
          "                                                            ^---- HERE"
        ],
        "script" : "ctx._source['@timestamp'] = OffsetDateTime.parse(ctx._source['@timestamp']).minusYears(20000).toString()",
        "lang" : "painless",
        "position" : {
          "offset" : 60,
          "start" : 0,
          "end" : 104
        }
      }
    ],
    "type" : "script_exception",
    "reason" : "runtime error",
    "script_stack" : [
      "java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2051)",
      "java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1953)",
      "java.base/java.time.OffsetDateTime.parse(OffsetDateTime.java:403)",
      "java.base/java.time.OffsetDateTime.parse(OffsetDateTime.java:388)",
      "ctx._source['@timestamp'] = OffsetDateTime.parse(ctx._source['@timestamp']).minusYears(20000).toString()",
      "                                                            ^---- HERE"
    ],
    "script" : "ctx._source['@timestamp'] = OffsetDateTime.parse(ctx._source['@timestamp']).minusYears(20000).toString()",
    "lang" : "painless",
    "position" : {
      "offset" : 60,
      "start" : 0,
      "end" : 104
    },
    "caused_by" : {
      "type" : "date_time_parse_exception",
      "reason" : "Text '22020-12-16T12:01:48.000000000+11:00' could not be parsed at index 0"
    }
  },
  "status" : 400
}

Is there any other way of subtracting 20000 years from a timestamp?

Thanks
Dan

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.