# Unusual timestamp data that needs fixing
**URL:** https://discuss.elastic.co/t/unusual-timestamp-data-that-needs-fixing/259209
**Category:** Elasticsearch
**Created:** [December 20, 2020, 11:54pm UTC](https://discuss.elastic.co/t/unusual-timestamp-data-that-needs-fixing/259209 "2020-12-20T23:54:18Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@dan.mackie](https://discuss.elastic.co/u/dan.mackie)
#### Post date: [December 20, 2020, 11:54pm UTC](https://discuss.elastic.co/t/unusual-timestamp-data-that-needs-fixing/259209/1 "2020-12-20T23:54:19Z")
Hello all,
Elastic 7.9.2
Kibana 7.9.2
I have an index that has somehow ended up with a date 20000 years in the future.
I thought at first that the `@timestamp` field had simply been misinterpreted as text by elastic, which was causing shard failure warnings in kibana, and so I created a new index-
```auto
POST _reindex?wait_for_completion=true
{
"source": {
"index": "openstack-22020.12.16"
},
"dest": {
"index": "openstack-22020.12.16-new"
}
}
```
Deleted the old index-
```auto
DELETE openstack-22020.12.16
```
Recreated the old index with a date/time format defined-
```auto
PUT openstack-22020.12.16
{
"mappings": {
"properties": {
"@timestamp": {
"type": "date",
},
```
(truncated for brevity)
However when I tried to reindex the `openstack-22020.12.16-new` back to my newly created `openstack-22020.12.16`, I get the following error.
```auto
{
"took" : 2,
"timed_out" : false,
"total" : 2,
"updated" : 0,
"created" : 0,
"deleted" : 0,
"batches" : 1,
"version_conflicts" : 0,
"noops" : 0,
"retries" : {
"bulk" : 0,
"search" : 0
},
"throttled_millis" : 0,
"requests_per_second" : -1.0,
"throttled_until_millis" : 0,
"failures" : [
{
"index" : "openstack-22020.12.16",
"type" : "_doc",
"id" : "ku0SaXYB7PbJS_CzFndk",
"cause" : {
"type" : "mapper_parsing_exception",
"reason" : "failed to parse field [@timestamp] of type [date] in document with id 'ku0SaXYB7PbJS_CzFndk'. Preview of field's value: '22020-12-16T12:01:48.000000000+11:00'",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "failed to parse date field [22020-12-16T12:01:48.000000000+11:00] with format [ordinal_date_time_no_millis]",
"caused_by" : {
"type" : "date_time_parse_exception",
"reason" : "Failed to parse with all enclosed parsers"
}
}
},
"status" : 400
},
{
"index" : "openstack-22020.12.16",
"type" : "_doc",
"id" : "IO8TaXYB7PbJS_CzFcL1",
"cause" : {
"type" : "mapper_parsing_exception",
"reason" : "failed to parse field [@timestamp] of type [date] in document with id 'IO8TaXYB7PbJS_CzFcL1'. Preview of field's value: '22020-12-16T12:03:09.000000000+11:00'",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "failed to parse date field [22020-12-16T12:03:09.000000000+11:00] with format [ordinal_date_time_no_millis]",
"caused_by" : {
"type" : "date_time_parse_exception",
"reason" : "Failed to parse with all enclosed parsers"
}
}
},
"status" : 400
}
]
}
```
A closer inspection of the date and the index name shows that somehow, this has come in with the year set to 22020, and I'm not sure how to get the year back to a sane number.
I had a look at this answer here-
> [@Update timestamp in all documents in an index](https://discuss.elastic.co/t/update-timestamp-in-all-documents-in-an-index/170540):
>
> we had a backlog of messages in kafa for filebeat coming from syslog with the following format which were from 2018, Dec 31 23:59:59 phc1803-hdfs-03 consul: 2018/12/31 23:59:59 [WARN] agent: Check 'service:timeseries-hdfs-zookeeper-leader' is now critical we noticed the issue in 2019 and when these documents ran the timestamp was set to "@timestamp": "2019-12-31T23:59:59.000Z" because the grok filter was taking in MM DD and setting a year based on when it was parsed. Since we noticed the iss…
However I was not able to successfully update the date using the script examples there.
I created a new "test" version of the index and made it writeable-
```auto
PUT openstack-22020.12.16-test/_settings
{
"index": {
"blocks.write": false
}
}
```
Then tried
```auto
POST openstack-22020.12.16-test/_update_by_query
{
"query": {
"match_all": {}
},
"script": {
"source": "ctx._source['@timestamp'] = OffsetDateTime.parse(ctx._source['@timestamp']).minusYears(20000).toString()"
}
}
```
I suspect this is because of how far out of range the time has become. The error is
```auto
{
"error" : {
"root_cause" : [
{
"type" : "script_exception",
"reason" : "runtime error",
"script_stack" : [
"java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2051)",
"java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1953)",
"java.base/java.time.OffsetDateTime.parse(OffsetDateTime.java:403)",
"java.base/java.time.OffsetDateTime.parse(OffsetDateTime.java:388)",
"ctx._source['@timestamp'] = OffsetDateTime.parse(ctx._source['@timestamp']).minusYears(20000).toString()",
" ^---- HERE"
],
"script" : "ctx._source['@timestamp'] = OffsetDateTime.parse(ctx._source['@timestamp']).minusYears(20000).toString()",
"lang" : "painless",
"position" : {
"offset" : 60,
"start" : 0,
"end" : 104
}
}
],
"type" : "script_exception",
"reason" : "runtime error",
"script_stack" : [
"java.base/java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:2051)",
"java.base/java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1953)",
"java.base/java.time.OffsetDateTime.parse(OffsetDateTime.java:403)",
"java.base/java.time.OffsetDateTime.parse(OffsetDateTime.java:388)",
"ctx._source['@timestamp'] = OffsetDateTime.parse(ctx._source['@timestamp']).minusYears(20000).toString()",
" ^---- HERE"
],
"script" : "ctx._source['@timestamp'] = OffsetDateTime.parse(ctx._source['@timestamp']).minusYears(20000).toString()",
"lang" : "painless",
"position" : {
"offset" : 60,
"start" : 0,
"end" : 104
},
"caused_by" : {
"type" : "date_time_parse_exception",
"reason" : "Text '22020-12-16T12:01:48.000000000+11:00' could not be parsed at index 0"
}
},
"status" : 400
}
```
Is there any other way of subtracting 20000 years from a timestamp?
Thanks
Dan
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 17, 2021, 11:54pm UTC](https://discuss.elastic.co/t/unusual-timestamp-data-that-needs-fixing/259209/2 "2021-01-17T23:54:19Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.