# Reindex daily-indices in a single command

**URL:** <https://discuss.elastic.co/t/reindex-daily-indices-in-a-single-command/128892>\
**Category:** Elasticsearch\
**Created:** [April 20, 2018, 2:22pm UTC](https://discuss.elastic.co/t/reindex-daily-indices-in-a-single-command/128892 "2018-04-20T14:22:03Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ryan\_Grannell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ryan_grannell/32/41712_2.png) [@Ryan\_Grannell](https://discuss.elastic.co/u/Ryan_Grannell)\
**Post date:** [April 20, 2018, 2:22pm UTC](https://discuss.elastic.co/t/reindex-daily-indices-in-a-single-command/128892/1 "2018-04-20T14:22:03Z")

</div>

Hi,

I've recently had to update the type-mapping for ~30 indices, changing several fields from text → keyword [1].

I now need to do this again, and am wondering if there is an easier method than the one shown below.

We currently have the frustrating proceedure of:

1. Posting our new index mapping matching the pattern '_primary-_'
2. Manually reindexing each "primary" index to a "swap" index with the new mapping applied

> ```
> primary-2018-04-04 → swap-primary-2018-04-04
> primary-2018-04-05 → swap-primary-2018-04-05
> .
> .
> .
> primary-2018-04-20 → swap-primary-2018-04-20
> 
> ```

1. Deleteing each "primary-\*" index
2. Post an alias mapping each "swap-primary-\*" back to its "primary-\*" name

**Firstly** , is there any equivalent single '\_reindex' command that will at map each "primary-_" index to a corresponding "swap-primary-_" index for us in one go? We could automate this using the Elasticsearch API's, but I'd rather not if there's a built-in way of doing this.

**Secondly** , is it possible to change mappings for text fields → keyword fields 'in place'? I understand that in-place mapping updates are generally forbade, but the raw data needed for this field is present in \_source. If not, I'd love to see in-place field reindexing based on \_source be added in the future.

Thanks for any help you can offer,  
Ryan

_[1] We don't use the default template-mapping as we have a worryingly large field-count, so having a '.keyword' duplicate for each text field would double this count. We do control this count using "enabled:false" & similar tools; I mention it only to point out duplicating fields under a different mapping would not solve this problem for us._

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [April 27, 2018, 4:16pm UTC](https://discuss.elastic.co/t/reindex-daily-indices-in-a-single-command/128892/2 "2018-04-27T16:16:00Z")

</div>

> [@Ryan\_Grannell](#):
>
> Firstly, is there any equivalent single '\_reindex' command that will at map each "primary-" index to a corresponding "swap-primary-" index for us in one go? We could automate this using the Elasticsearch API's, but I'd rather not if there's a built-in way of doing this.

Afraid not. You'll have to do the dance (new index with appropriate mappings, reindex, delete, swap alias) each time. It's pretty straightforward to automate or wrap in a script, but it's not something we provide out of the box at the moment.

> [@Ryan\_Grannell](#):
>
> Secondly, is it possible to change mappings for text fields → keyword fields 'in place'? I understand that in-place mapping updates are generally forbade, but the raw data needed for this field is present in \_source. If not, I'd love to see in-place field reindexing based on \_source be added in the future.

Also not possible. It's a good thought and not immediately obvious why we forbid it. The main issue isn't the source data... like you said, it's available (unless you've disabled source, which could be a different issue).

The problem is that Lucene only knows how to map one data type to one field. During the "update in place", there's no way to overwrite the existing field's internal data structure with the `keyword` in place of the `text`.

It also poses a complicated question of how to handle searches that are going against a "mixed" index. Some data would need to be analyzed, other wouldn't. The keyword field may have normalizers that are not compatible with the text's analyzer. Term and doc frequencies will get super messed up during the transition and completely ruin scoring. Etc etc 🙂

All in all, it's a very complicated situation that doesn't have a lot of immediately obvious "correct" answers, but does come with a lot of baggage and edge-cases.

It might be possible in the future as things continue to evolve, but at least for right now it isn't. Hope that helps provide a bit of background!

---

<div class="post-metadata">

**Author:** ![Ryan\_Grannell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ryan_grannell/32/41712_2.png) [@Ryan\_Grannell](https://discuss.elastic.co/u/Ryan_Grannell)\
**Post date:** [May 3, 2018, 10:19am UTC](https://discuss.elastic.co/t/reindex-daily-indices-in-a-single-command/128892/3 "2018-05-03T10:19:54Z")

</div>

Thanks Zachary, I appreciate the answers to both questions.

---

<div class="post-metadata">

**Author:** ![jetnet](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@jetnet](https://discuss.elastic.co/u/jetnet)\
**Post date:** [May 26, 2018, 11:28am UTC](https://discuss.elastic.co/t/reindex-daily-indices-in-a-single-command/128892/4 "2018-05-26T11:28:02Z")

</div>

I apologize for warming up this topic, but I'd like to ask you guys - what do you think of this:

> [@polyfractal](#):
>
> > Firstly, is there any equivalent single '\_reindex' command that will at map each "primary-" index to a corresponding "swap-primary-" index for us in one go? We could automate this using the Elasticsearch API's, but I'd rather not if there's a built-in way of doing this.
> 
> Afraid not. You'll have to do the dance (new index with appropriate mappings, reindex, delete, swap alias) each time. It's pretty straightforward to automate or wrap in a script, but it's not something we provide out of the box at the moment.

I was able to reindex daily indices (`net-YYYY.mm.dd`) as following:

- create a template with the mapping/settings for the new indices

```auto
PUT _template/net-6
{
 "index_patterns": ["net-6-*"],
 "order" : 1000,
 "settings": {
   "index": {
     "number_of_replicas": 0,
     "number_of_shards": 1
   }
 },
 "mappings": {
   "_doc": {
...

```

- \_reindes API with a painless script to create [daily indices](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#_reindex_daily_indices) based on the template above

```auto
POST _reindex?wait_for_completion=false
{
 "source": {
   "index": "net-20*",
   "type": "net"
 },
 "dest": {
   "index": "net-6",
   "type": "_doc"
 },
 "script": {
 "source": """
       ctx._index = "net-6-" + (ctx._index.substring('net-'.length(), ctx._index.length()));
       // some other stuff

```

P.S. as you can see, I dont really like dancing )

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [May 31, 2018, 2:20pm UTC](https://discuss.elastic.co/t/reindex-daily-indices-in-a-single-command/128892/5 "2018-05-31T14:20:26Z")

</div>

Ah nice, that's a clever use of the reindex script. Filing that away for future reference myself 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 28, 2018, 2:20pm UTC](https://discuss.elastic.co/t/reindex-daily-indices-in-a-single-command/128892/6 "2018-06-28T14:20:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.