# How to reindex an ES index **URL:** \ **Category:** Elasticsearch\ **Created:** [June 21, 2011, 9:38am UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669 "2011-06-21T09:38:59Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![dpilato](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpilato/32/2449_2.png) [@dpilato](https://discuss.elastic.co/u/dpilato)\ **Post date:** [June 21, 2011, 9:38am UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669/1 "2011-06-21T09:38:59Z")
Hi, Here is my usecase : I have an ES index (myindex) In this index, I have one document type (myindex/mydoc) I have defined a mapping for this type (myindex/mydoc/\_mapping) I put many documents in this index (with default options so JSon source id stored in \_source). I want now to modify my mapping and ask to ES to reindex all my documents, based on the \_source. Is there any way to do that ? Do I have to create an ES river plugin to be able for example to reindex data in a new index based on the \_source field of documents in the first index ? Is it the purpose of the [\_refresh API](http://www.elasticsearch.org/guide/reference/api/admin-indices-refresh.html) ? I don't think so. Thanks for your feedbacks/advices David ---
**Author:** ![dpilato](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpilato/32/2449_2.png) [@dpilato](https://discuss.elastic.co/u/dpilato)\ **Post date:** [June 21, 2011, 9:43am UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669/2 "2011-06-21T09:43:48Z")
BTW, is it possible to remove spams from fashionalwallet as it make a mess in the web view of the mailing list ! [http://elasticsearch-users.115913.n3.nabble.com/](http://elasticsearch-users.115913.n3.nabble.com/) Thanks... ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [June 21, 2011, 9:45am UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669/3 "2011-06-21T09:45:03Z")
Hi David > I want now to modify my mapping and ask to ES to reindex all my documents, > based on the \_source. > > Is there any way to do that ? The easiest way to do this is to script it using whatever API you're using to access ES ie: - create new index with new mapping - do a scrolled match\_all search with search\_type=scan of the old index - while the scrolled search is still returning results: - change the \_source if you need to - bulk\_index eg 1,000 docs to the new index - done I think the python ruby and Perl APIs all provide convenience methods to help you with the process, eg: [http://blogs.perl.org/users/clinton\_gormley/2011/04/elasticsearchpm-v036-now-with-extra-sugar.html](http://blogs.perl.org/users/clinton_gormley/2011/04/elasticsearchpm-v036-now-with-extra-sugar.html) clint ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 21, 2011, 6:00pm UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669/4 "2011-06-21T18:00:44Z")
Many thanks Clint for this. I will try it (just have to check if I have perl on my server 😉 !) Do you think that it could make sense of creating a ReplicateRiver to do that job with simple curl calls ? As simple as we can find the replicate function in CouchDB... Thanks again, David. -----Message d'origine----- De : Clinton Gormley [[mailto:clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk)] Envoyé : mardi 21 juin 2011 11:45 À : [users@elasticsearch.com](mailto:users@elasticsearch.com) Objet : Re: How to reindex an ES index Hi David > I want now to modify my mapping and ask to ES to reindex all my documents, > based on the \_source. > > Is there any way to do that ? The easiest way to do this is to script it using whatever API you're using to access ES ie: - create new index with new mapping - do a scrolled match\_all search with search\_type=scan of the old index - while the scrolled search is still returning results: - change the \_source if you need to - bulk\_index eg 1,000 docs to the new index - done I think the python ruby and Perl APIs all provide convenience methods to help you with the process, eg: [http://blogs.perl.org/users/clinton\_gormley/2011/04/elasticsearchpm-v036-now-with-extra-sugar.html](http://blogs.perl.org/users/clinton_gormley/2011/04/elasticsearchpm-v036-now-with-extra-sugar.html) clint ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [June 23, 2011, 11:43am UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669/5 "2011-06-23T11:43:46Z")
Why do you need a river for that? You can write it using the Java API as well. On Tuesday, June 21, 2011 at 9:00 PM, David Pilato wrote: > Many thanks Clint for this. I will try it (just have to check if I have perl on my server 😉 !) > > Do you think that it could make sense of creating a ReplicateRiver to do that job with simple curl calls ? > As simple as we can find the replicate function in CouchDB... > > Thanks again, > David. > > -----Message d'origine----- > De : Clinton Gormley [[mailto:clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk)] > Envoyé : mardi 21 juin 2011 11:45 > À : [users@elasticsearch.com](mailto:users@elasticsearch.com) ([mailto:users@elasticsearch.com](mailto:users@elasticsearch.com)) > Objet : Re: How to reindex an ES index > > Hi David > > > I want now to modify my mapping and ask to ES to reindex all my documents, > > based on the \_source. > > > > Is there any way to do that ? > > The easiest way to do this is to script it using whatever API you're > using to access ES ie: > > - create new index with new mapping > - do a scrolled match\_all search with search\_type=scan of the old index > - while the scrolled search is still returning results: > - change the \_source if you need to > - bulk\_index eg 1,000 docs to the new index > - done > > I think the python ruby and Perl APIs all provide convenience methods to > help you with the process, eg: > > [ElasticSearch.pm v0.36, now with extra sugar | Clinton Gormley [blogs.perl.org]](http://blogs.perl.org/users/clinton_gormley/2011/04/elasticsearchpm-v036-now-with-extra-sugar.html) > > clint ---
**Author:** ![dpilato](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpilato/32/2449_2.png) [@dpilato](https://discuss.elastic.co/u/dpilato)\ **Post date:** [June 23, 2011, 12:56pm UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669/6 "2011-06-23T12:56:30Z")
Why do you need a river for that? You can write it using the Java API as well. Sure. I was thinking of a simple admin tool to reindex all datas in another ES cluster/index/type without a single line of code. Nonsense ? ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [June 23, 2011, 1:12pm UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669/7 "2011-06-23T13:12:21Z")
> Sure. > > I was thinking of a simple admin tool to reindex all datas in another ES > cluster/index/type without a single line of code. > > Nonsense ? Why are you reindexing? Either you want to change the mapping, or the index settings (number of shards, analyzers) or you want to change the data itself. In order to do this, you need be able to hook into various phases of the reindexing process, and apply some logic. This, by its very nature, implies some custom code. Typically, this is easier to implement in the client code clint ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 23, 2011, 9:25pm UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669/8 "2011-06-23T21:25:59Z")
> Why are you reindexing? > Because, I need to change my mapping (index existing fields) > Or I want to use another analyzer or to boost some fields. As i don't change my document itself, ES has already it in the \_source field. I just want to change my mapping and apply it to previous indexed docs. If I merge the new mapping with the old one (if there is no conflict), I don't think that ES parse automatically previous docs and reindex them. Am I wrong ? Is there a best practice to reindex existing datas ? Cheers David 😉 Le 23 juin 2011 à 15:12, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk) a écrit : > > Sure. > > > > I was thinking of a simple admin tool to reindex all datas in another ES > > cluster/index/type without a single line of code. > > > > Nonsense ? > > Why are you reindexing? > > Either you want to change the mapping, or the index settings (number of > shards, analyzers) or you want to change the data itself. > > In order to do this, you need be able to hook into various phases of the > reindexing process, and apply some logic. This, by its very nature, > implies some custom code. > > Typically, this is easier to implement in the client code > > clint ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [June 24, 2011, 9:41am UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669/9 "2011-06-24T09:41:39Z")
For cases where you can simply take the \_source and index it into a new index (possibly with different mappings), then it makes sense. Having something built in elasticsearch for that does make sense, definitely. Its not a high priority since with the scan API, one can implement it on the client side. On Friday, June 24, 2011 at 12:25 AM, David Pilato wrote: > > Why are you reindexing? > > Because, I need to change my mapping (index existing fields) > > Or I want to use another analyzer or to boost some fields. > > As i don't change my document itself, ES has already it in the \_source field. I just want to change my mapping and apply it to previous indexed docs. > > If I merge the new mapping with the old one (if there is no conflict), I don't think that ES parse automatically previous docs and reindex them. > > Am I wrong ? > > Is there a best practice to reindex existing datas ? > > Cheers > David 😉 > > Le 23 juin 2011 à 15:12, Clinton Gormley \<[clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk) ([mailto:clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk))\> a écrit : > > > > Sure. > > > > > > I was thinking of a simple admin tool to reindex all datas in another ES > > > cluster/index/type without a single line of code. > > > > > > Nonsense ? > > > > Why are you reindexing? > > > > Either you want to change the mapping, or the index settings (number of > > shards, analyzers) or you want to change the data itself. > > > > In order to do this, you need be able to hook into various phases of the > > reindexing process, and apply some logic. This, by its very nature, > > implies some custom code. > > > > Typically, this is easier to implement in the client code > > > > clint ---
**Author:** ![James\_Cook](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@James\_Cook](https://discuss.elastic.co/u/James_Cook)\ **Post date:** [June 27, 2011, 5:21pm UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669/10 "2011-06-27T17:21:22Z")
On Thu, Jun 23, 2011 at 9:12 AM, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk)wrote: > > Sure. > > > > I was thinking of a simple admin tool to reindex all datas in another ES > > cluster/index/type without a single line of code. > > > > Nonsense ? > > Why are you reindexing? > > Either you want to change the mapping, or the index settings (number of > shards, analyzers) or you want to change the data itself. Or moving to a newer version of ES requires it. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 30, 2011, 8:46am UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669/11 "2011-06-30T08:46:20Z")
For cases where you can simply take the \_source and index it into a new index (possibly with different mappings), then it makes sense. Having something built in elasticsearch for that does make sense, definitely. Its not a high priority since with the scan API, one can implement it on the client side. Thanks. Just opened a Feature Request for that here : [https://github.com/elasticsearch/elasticsearch/issues/1077](https://github.com/elasticsearch/elasticsearch/issues/1077) I will "try" to code it as writing rivers seems to be "easy"... ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:02am UTC](https://discuss.elastic.co/t/how-to-reindex-an-es-index/4669/12 "2017-07-06T04:02:12Z")