# Date field type problem in reindexing

**URL:** <https://discuss.elastic.co/t/date-field-type-problem-in-reindexing/212865>\
**Category:** Elasticsearch\
**Created:** [December 23, 2019, 3:56pm UTC](https://discuss.elastic.co/t/date-field-type-problem-in-reindexing/212865 "2019-12-23T15:56:41Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sina\_Gholami](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sina_gholami/32/59877_2.png) [@Sina\_Gholami](https://discuss.elastic.co/u/Sina_Gholami)\
**Post date:** [December 23, 2019, 3:56pm UTC](https://discuss.elastic.co/t/date-field-type-problem-in-reindexing/212865/1 "2019-12-23T15:56:41Z")

</div>

I am trying to merge multiple indices with same fields in Elasticsearch by using \_reindex API as follow:  
POST \_reindex  
{  
"source": {  
"index": ["a","b"]

},  
"dest": {  
"index": "ab"  
}  
}

my problem is the date field. In "ab" index the date field becomes string and I cannot understand why. I used the following command in \_reindex and tried to create and set mapping for index "ab" before \_reindex, but the problem is not resolved.

"script": {  
"source": "ctx.\_source.date = new SimpleDateFormat('yyyy-MM-dd HH:mm:ss').parse(ctx.\_source.date);"  
}

PUT /data  
{  
"settings": {  
"number\_of\_shards": 1,  
"number\_of\_replicas": 1  
},  
"mappings": {  
"properties" : {  
"cashtags" : {  
"type" : "keyword"  
},  
"conversation\_id" : {  
"type" : "long"  
},  
"created\_at" : {  
"type" : "long"  
},  
"date" : {  
"type": "date",  
"format" : "yyyy-MM-dd HH:mm:ss"  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![pgomulka](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pgomulka/32/45766_2.png) [@pgomulka](https://discuss.elastic.co/u/pgomulka)\
**Post date:** [December 31, 2019, 10:13am UTC](https://discuss.elastic.co/t/date-field-type-problem-in-reindexing/212865/2 "2019-12-31T10:13:58Z")

</div>

would you be able to share example date field values?

---

<div class="post-metadata">

**Author:** ![Sina\_Gholami](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sina_gholami/32/59877_2.png) [@Sina\_Gholami](https://discuss.elastic.co/u/Sina_Gholami)\
**Post date:** [January 2, 2020, 11:46am UTC](https://discuss.elastic.co/t/date-field-type-problem-in-reindexing/212865/3 "2020-01-02T11:46:43Z")

</div>

they are tweets which were downloaded by using twint library in python.  
here are the example:

id:  
1,203,786,367,391,674,368  
conversation\_id:  
1,203,786,367,391,674,368  
created\_at:  
1,575,840,009,000  
date:  
Dec 8, 2019 @ 23:20:09.000  
timezone:  
CET  
place:  
tweet:  
واقعا چجوری روتون میشه از انتخابات و رای دادن حرف بزنید، اونم زمانی که صندوق رای جمهوری اسلامی بوی خون گرفته . . . #انتخابات #انتخابات\_مجلس #تحریم\_خبری\_انتخابات #آبان98 #آبان\_خونین #پویا\_بختیاری  
hashtags:  
#انتخابات, #انتخابات\_مجلس, #تحریم\_خبری\_انتخابات, #آبان98, #آبان\_خونین, #پویا\_بختیاری  
cashtags:  
user\_id\_str:  
844586778942230529  
username:  
PHNTOMCAT  
name:  
مرد تنهای شب  
day:  
7  
hour:  
22  
link:  
[https://twitter.com/PHNTOMCAT/status/1203786367391674370](https://twitter.com/PHNTOMCAT/status/1203786367391674370)  
retweet:  
false  
essid:  
nlikes:  
3  
nreplies:  
1  
nretweets:  
0  
quote\_url:  
video:  
0  
search:  
رای دادن مجلس  
near:  
reply\_to:  
{ "user\_id": "844586778942230529", "username": "PHNTOMCAT" }  
_id:  
1203786367391674370\_raw_  
\_type:  
\_doc  
\_index:  
votingparliament  
\_score:  
1

---

<div class="post-metadata">

**Author:** ![pgomulka](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pgomulka/32/45766_2.png) [@pgomulka](https://discuss.elastic.co/u/pgomulka)\
**Post date:** [January 3, 2020, 7:47am UTC](https://discuss.elastic.co/t/date-field-type-problem-in-reindexing/212865/4 "2020-01-03T07:47:34Z")

</div>

> [@Sina\_Gholami](#):
>
> date:  
> Dec 8, 2019 @ 23:20:09.000

from what I see you have a custom date format on that field. Can you show a mapping for that field on index "a" and "b" ?  
you should use the same mapping on your index "ab" or use a script which would parse with pattern "MMM dd, uuuu @ hh:mm:ss.SSS"  
I assumed you are using ES version 7+. if not use YYYY instead of uuuu

---

<div class="post-metadata">

**Author:** ![Sina\_Gholami](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sina_gholami/32/59877_2.png) [@Sina\_Gholami](https://discuss.elastic.co/u/Sina_Gholami)\
**Post date:** [January 5, 2020, 10:33am UTC](https://discuss.elastic.co/t/date-field-type-problem-in-reindexing/212865/5 "2020-01-05T10:33:53Z")

</div>

Yes I am using ES 7.4.2.  
Since "a" and "b" are exactly the same I just print the "a" mapping here. I got the mapping by  
GET /a/\_mapping:  
{  
"a" : {  
"mappings" : {  
"properties" : {  
"cashtags" : {  
"type" : "keyword",  
"normalizer" : "hashtag\_normalizer"  
},  
"conversation\_id" : {  
"type" : "long"  
},  
"created\_at" : {  
"type" : "long"  
},  
"date" : {  
"type" : "date",  
"format" : "yyyy-MM-dd HH:mm:ss"  
},  
"day" : {  
"type" : "integer"  
},  
"essid" : {  
"type" : "keyword"  
},  
"geo\_near" : {  
"type" : "geo\_point"  
},  
"geo\_tweet" : {  
"type" : "geo\_point"  
},  
"hashtags" : {  
"type" : "keyword",  
"normalizer" : "hashtag\_normalizer"  
},  
"hour" : {  
"type" : "integer"  
},  
"id" : {  
"type" : "long"  
},  
"link" : {  
"type" : "text"  
},  
"location" : {  
"type" : "keyword"  
},  
"mentions" : {  
"type" : "keyword",  
"normalizer" : "hashtag\_normalizer"  
},  
"name" : {  
"type" : "text"  
},  
"near" : {  
"type" : "text"  
},  
"nlikes" : {  
"type" : "integer"  
},  
"nreplies" : {  
"type" : "integer"  
},  
"nretweets" : {  
"type" : "integer"  
},  
"photos" : {  
"type" : "text"  
},  
"place" : {  
"type" : "keyword"  
},  
"profile\_image\_url" : {  
"type" : "text"  
},  
"quote\_url" : {  
"type" : "text"  
},  
"reply\_to" : {  
"type" : "nested",  
"properties" : {  
"user\_id" : {  
"type" : "keyword"  
},  
"username" : {  
"type" : "keyword"  
}  
}  
},  
"retweet" : {  
"type" : "text"  
},  
"retweet\_date" : {  
"type" : "date",  
"format" : "yyyy-MM-dd HH:mm:ss"  
},  
"retweet\_id" : {  
"type" : "keyword"  
},  
"search" : {  
"type" : "text"  
},  
"source" : {  
"type" : "keyword"  
},  
"timezone" : {  
"type" : "keyword"  
},  
"trans\_dest" : {  
"type" : "keyword"  
},  
"trans\_src" : {  
"type" : "keyword"  
},  
"translate" : {  
"type" : "text"  
},  
"tweet" : {  
"type" : "text"  
},  
"urls" : {  
"type" : "keyword"  
},  
"user\_id\_str" : {  
"type" : "keyword"  
},  
"user\_rt" : {  
"type" : "keyword"  
},  
"user\_rt\_id" : {  
"type" : "keyword"  
},  
"username" : {  
"type" : "keyword",  
"normalizer" : "hashtag\_normalizer"  
},  
"video" : {  
"type" : "integer"  
}  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![Sina\_Gholami](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sina_gholami/32/59877_2.png) [@Sina\_Gholami](https://discuss.elastic.co/u/Sina_Gholami)\
**Post date:** [January 6, 2020, 11:23am UTC](https://discuss.elastic.co/t/date-field-type-problem-in-reindexing/212865/6 "2020-01-06T11:23:28Z")

</div>

Finally, I managed to solve this silly problem just by removing the saved object "ab" in kibana. no need for "script" in \_reindex and the PUT /ab {..."mappings":{...}} is enough.

---

<div class="post-metadata">

**Author:** ![pgomulka](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pgomulka/32/45766_2.png) [@pgomulka](https://discuss.elastic.co/u/pgomulka)\
**Post date:** [January 16, 2020, 8:08am UTC](https://discuss.elastic.co/t/date-field-type-problem-in-reindexing/212865/7 "2020-01-16T08:08:43Z")

</div>

@Sina_Gholami looks like the mapping on a (and b) is the same as on the `ab` you created. It is surprising to see date in this format Dec 8, 2019 @ 23:20:09.000  
Maybe it wasn't the date format it was actually sent to elasticsearch. I expect it would only parse `yyyy-MM-dd HH:mm:ss`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 13, 2020, 8:08am UTC](https://discuss.elastic.co/t/date-field-type-problem-in-reindexing/212865/8 "2020-02-13T08:08:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
