# Index deleted when putMapping is used twice

**URL:** https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654
**Category:** Elasticsearch
**Created:** [October 21, 2011, 7:23am UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654 "2011-10-21T07:23:21Z")
**Posts on this page:** 16
**Page:** 1

<div class="post-metadata">

### Author: ![Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre/32/3014_2.png) [@Alexandre](https://discuss.elastic.co/u/Alexandre)
#### Post date: [October 21, 2011, 7:23am UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/1 "2011-10-21T07:23:21Z")

</div>

Hi !

I am developping a small library that integrates Morphia (MongoDB) and  
ElasticSearch using the ES java API.  
During application startup I use Annotated Entities to generate the ES  
index Mapping.

Everything works pretty fine and I am able to create the index, then  
analyzers and the type mappings properly . Indexing works great too.

However I have noted that when restarting the application and going  
through the index mapping generation process again (using putMapping  
API) the content of the index gets deleted.

I do perform a check to see if the index already exists to avoid  
recreating it, however, in my Library the mappings are automatically  
uploaded each time the app starts which maybe is the problem ...

Is it a normal behavior of ES? Is the content wiped away when type  
mappings are redefined, even if they are exactly the same as the  
prevoius ones?  
Is there a way to get existing mappings and compare them to the ones  
I'm about to upload to avoid this problem?

Hope I'm clear enough 🙂

Thanks for your help

Cheers !

Alexandre

---

<div class="post-metadata">

### Author: ![Karussell1](https://avatars.discourse-cdn.com/v4/letter/k/50afbb/32.png) [@Karussell1](https://discuss.elastic.co/u/Karussell1)
#### Post date: [October 21, 2011, 11:05am UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/2 "2011-10-21T11:05:14Z")

</div>

the putMapping api should merge the mapping of the fields:

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

but content should stay or are you doing a delete mapping reqest?

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

Can you track down the issues to some curl or java statements?

and yes, you can also get the current mapping

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

I would recommend to reindex the data if the mapping changed otherwise  
you possibly get wrong mapping merges and incorrect indexed fields...

Regards,  
Peter.

On 21 Okt., 09:23, Alexandre [azli...@gmail.com](mailto:azli...@gmail.com) wrote:

> Hi !
> 
> I am developping a small library that integrates Morphia (MongoDB) and  
> Elasticsearch using the ES java API.  
> During application startup I use Annotated Entities to generate the ES  
> index Mapping.
> 
> Everything works pretty fine and I am able to create the index, then  
> analyzers and the type mappings properly . Indexing works great too.
> 
> However I have noted that when restarting the application and going  
> through the index mapping generation process again (using putMapping  
> API) the content of the index gets deleted.
> 
> I do perform a check to see if the index already exists to avoid  
> recreating it, however, in my Library the mappings are automatically  
> uploaded each time the app starts which maybe is the problem ...
> 
> Is it a normal behavior of ES? Is the content wiped away when type  
> mappings are redefined, even if they are exactly the same as the  
> prevoius ones?  
> Is there a way to get existing mappings and compare them to the ones  
> I'm about to upload to avoid this problem?
> 
> Hope I'm clear enough 🙂
> 
> Thanks for your help
> 
> Cheers !
> 
> Alexandre

---

<div class="post-metadata">

### Author: ![Karussell1](https://avatars.discourse-cdn.com/v4/letter/k/50afbb/32.png) [@Karussell1](https://discuss.elastic.co/u/Karussell1)
#### Post date: [October 21, 2011, 11:08am UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/3 "2011-10-21T11:08:45Z")

</div>

> I would recommend to reindex the data

only if you changed or removed a field ... for new fields updating  
without reindexing should be fine

---

<div class="post-metadata">

### Author: ![Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre/32/3014_2.png) [@Alexandre](https://discuss.elastic.co/u/Alexandre)
#### Post date: [October 21, 2011, 12:36pm UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/4 "2011-10-21T12:36:34Z")

</div>

> Can you track down the issues to some curl or java statements?

Sure !

Create index is called at framework startup :

> <https://gist.github.com/anonymous/1303673>

then each entity is analyzed and a mapping is generated using an  
XContentBuilder. Here is an example of a generated mapping :

> <https://gist.github.com/anonymous/1303707>

and here are the instructions used to put mapping in es :

> <https://gist.github.com/anonymous/1303712>

Finally insertion of random data .

> <https://gist.github.com/anonymous/1303716>

Works great on first launch

BUT, if I relaunch it without indexing random stuff ... my previous  
indexed content has dissapeard from ES ...

Thanks again for you help

---

<div class="post-metadata">

### Author: ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [October 21, 2011, 6:25pm UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/5 "2011-10-21T18:25:31Z")

</div>

Thanks for the gists, one way to simulate it faster is if you have a simpel  
"main" program that uses the APIs to shows it. Can you do it?

On Fri, Oct 21, 2011 at 2:36 PM, Alexandre [azlist1@gmail.com](mailto:azlist1@gmail.com) wrote:

> > Can you track down the issues to some curl or java statements?
> 
> Sure !
> 
> Create index is called at framework startup :
> 
> [gist:1303673 · GitHub](https://gist.github.com/1303673)
> 
> then each entity is analyzed and a mapping is generated using an  
> XContentBuilder. Here is an example of a generated mapping :
> 
> [gist:1303707 · GitHub](https://gist.github.com/1303707)
> 
> and here are the instructions used to put mapping in es :
> 
> [gist:1303712 · GitHub](https://gist.github.com/1303712)
> 
> Finally insertion of random data .
> 
> [gist:1303716 · GitHub](https://gist.github.com/1303716)
> 
> Works great on first launch
> 
> BUT, if I relaunch it without indexing random stuff ... my previous  
> indexed content has dissapeard from ES ...
> 
> Thanks again for you help

---

<div class="post-metadata">

### Author: ![Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre/32/3014_2.png) [@Alexandre](https://discuss.elastic.co/u/Alexandre)
#### Post date: [October 21, 2011, 10:27pm UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/6 "2011-10-21T22:27:07Z")

</div>

Hey !

Thanks for following up !

Here's a full test that you can run.

> <https://gist.github.com/anonymous/1305141>

You will jsut need a small lorem ipsum lib to generate the random  
content and its available here :

[http://loremipsum.sourceforge.net/](http://loremipsum.sourceforge.net/)

As you will see content is wiped out of the index ... -\_-

Let me know what you think.

Regards,

Alexandre

---

<div class="post-metadata">

### Author: ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [October 22, 2011, 1:03am UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/7 "2011-10-22T01:03:18Z")

</div>

Hey,

I took your code and ran it (on 0.17.9, and master), and it works for me.  
Started both a 1 node cluster, and a 2 nodes cluster, and getting the  
correct doc count on both cases. Any more data on your usage?

Btw, few notes on elasticsearch usage. You don't have to close an index to  
put the mappings. Also, you can provide the mappings on index creation time  
(so you remove the need to have the put mapping call). Last, if you want to  
make things searchable "right away", you can simply call the refresh API.

On Sat, Oct 22, 2011 at 12:27 AM, Alexandre [azlist1@gmail.com](mailto:azlist1@gmail.com) wrote:

> Hey !
> 
> Thanks for following up !
> 
> Here's a full test that you can run.
> 
> [gist:1305141 · GitHub](https://gist.github.com/1305141)
> 
> You will jsut need a small lorem ipsum lib to generate the random  
> content and its available here :
> 
> [http://loremipsum.sourceforge.net/](http://loremipsum.sourceforge.net/)
> 
> As you will see content is wiped out of the index ... -\_-
> 
> Let me know what you think.
> 
> Regards,
> 
> Alexandre

---

<div class="post-metadata">

### Author: ![Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre/32/3014_2.png) [@Alexandre](https://discuss.elastic.co/u/Alexandre)
#### Post date: [October 22, 2011, 12:13pm UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/8 "2011-10-22T12:13:26Z")

</div>

Hi,

I just upgraded to 0.17.9 and re-ran the test I gave you... and the  
problem was still here for me ...

Then I also modified my program a bit to take some of your comments  
into account such as using refresh instead of sleep and NOT taking the  
index down before putMapping.

And swoosh!! problem seems to have disappeared ...

I don't know what the source of the problem was but maybe there is a  
hidden bug to investigate ...

Anyway thank you very much for your help.  
I'll probably be bugging you again soon ...

Have a nice day !

---

<div class="post-metadata">

### Author: ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [October 22, 2011, 8:42pm UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/9 "2011-10-22T20:42:32Z")

</div>

Heya, it still should not happen. Can you give more details on your  
deployment? How big is the cluster (how many nodes), configuration of it.

On Sat, Oct 22, 2011 at 2:13 PM, Alexandre [azlist1@gmail.com](mailto:azlist1@gmail.com) wrote:

> Hi,
> 
> I just upgraded to 0.17.9 and re-ran the test I gave you... and the  
> problem was still here for me ...
> 
> Then I also modified my program a bit to take some of your comments  
> into account such as using refresh instead of sleep and NOT taking the  
> index down before putMapping.
> 
> And swoosh!! problem seems to have disappeared ...
> 
> I don't know what the source of the problem was but maybe there is a  
> hidden bug to investigate ...
> 
> Anyway thank you very much for your help.  
> I'll probably be bugging you again soon ...
> 
> Have a nice day !

---

<div class="post-metadata">

### Author: ![Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre/32/3014_2.png) [@Alexandre](https://discuss.elastic.co/u/Alexandre)
#### Post date: [October 22, 2011, 9:41pm UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/10 "2011-10-22T21:41:56Z")

</div>

This is a dev / test server so I only have 1 node + 1 shard + 0  
replicas

- 1 client connecting to it on the same machine.

---

<div class="post-metadata">

### Author: ![Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre/32/3014_2.png) [@Alexandre](https://discuss.elastic.co/u/Alexandre)
#### Post date: [October 22, 2011, 9:42pm UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/11 "2011-10-22T21:42:58Z")

</div>

Configuration is the default config I think ... I didnt change  
anything as far as I can remember.

---

<div class="post-metadata">

### Author: ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [October 23, 2011, 12:12am UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/12 "2011-10-23T00:12:00Z")

</div>

tried to recreate it again with a single node, 1 shard 0 replica index, and  
still seems to work.. . Maybe you can check what config you used?

On Sat, Oct 22, 2011 at 11:42 PM, Alexandre [azlist1@gmail.com](mailto:azlist1@gmail.com) wrote:

> Configuration is the default config I think ... I didnt change  
> anything as far as I can remember.

---

<div class="post-metadata">

### Author: ![Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre/32/3014_2.png) [@Alexandre](https://discuss.elastic.co/u/Alexandre)
#### Post date: [October 23, 2011, 12:48am UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/13 "2011-10-23T00:48:03Z")

</div>

Ok so first of all here are all the configuration infos I can find in  
my GUI :

> <https://gist.github.com/anonymous/1306691>
>
> There are more than three files. show original

Next I noticed something strange in my console that occurs when  
triggering the createIndex(true) method of the test I gave you.  
I don0t if its related to the same problem ...

In the logs I get the following :

Exception caught on netty layer [[id: 0x552297d7, /192.168.0.52:55124  
=\> /192.168.0.52:9301]]

here is the full log of what happened just when creating the index.

> <https://gist.github.com/anonymous/1306700>

---

<div class="post-metadata">

### Author: ![Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre/32/3014_2.png) [@Alexandre](https://discuss.elastic.co/u/Alexandre)
#### Post date: [October 23, 2011, 12:59am UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/14 "2011-10-23T00:59:48Z")

</div>

Plus I am now sure that the index wiping problem ONLY occurs when the  
index is taken down before putMapping calls in the test ...

If I comment out the close + open instructions the problem does not  
occur ...

//  
getClient().admin().indices().prepareClose(globalIndexname).execute().actionGet();

//  
getClient().admin().indices().prepareOpen(globalIndexname).execute().actionGet();

---

<div class="post-metadata">

### Author: ![Alexandre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexandre/32/3014_2.png) [@Alexandre](https://discuss.elastic.co/u/Alexandre)
#### Post date: [October 24, 2011, 11:38am UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/15 "2011-10-24T11:38:18Z")

</div>

Hey !

Just to let you know :

I just tested this again today on a different machine (32bit vista)  
with a fresh install of ES 0.17.9 with no configuration changes from  
what was downloaded and the weird index wiping still occurred.

When the index is taken down before putMapping (even though its not  
necessary), the first putMapping call works fine and documents are  
indexed but the seccond putMapping call (with same mapping) wipes  
the index.

When the index is NOT taken down both putMapping call work fine and  
the documents are still there after both calls.

Hope this helps you.

Cheers !

Alex

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:51am UTC](https://discuss.elastic.co/t/index-deleted-when-putmapping-is-used-twice/5654/16 "2017-07-06T03:51:04Z")

</div>


