Index deleted when putMapping is used twice

Alexandre · October 21, 2011, 7:23am

Hi !

I am developping a small library that integrates Morphia (MongoDB) and
ElasticSearch using the ES java API.
During application startup I use Annotated Entities to generate the ES
index Mapping.

Everything works pretty fine and I am able to create the index, then
analyzers and the type mappings properly . Indexing works great too.

However I have noted that when restarting the application and going
through the index mapping generation process again (using putMapping
API) the content of the index gets deleted.

I do perform a check to see if the index already exists to avoid
recreating it, however, in my Library the mappings are automatically
uploaded each time the app starts which maybe is the problem ...

Is it a normal behavior of ES? Is the content wiped away when type
mappings are redefined, even if they are exactly the same as the
prevoius ones?
Is there a way to get existing mappings and compare them to the ones
I'm about to upload to avoid this problem?

Hope I'm clear enough

Thanks for your help

Cheers !

Alexandre

Karussell1 · October 21, 2011, 11:05am

the putMapping api should merge the mapping of the fields:

but content should stay or are you doing a delete mapping reqest?

Can you track down the issues to some curl or java statements?

and yes, you can also get the current mapping

I would recommend to reindex the data if the mapping changed otherwise
you possibly get wrong mapping merges and incorrect indexed fields...

Regards,
Peter.

On 21 Okt., 09:23, Alexandre azli...@gmail.com wrote:

Hi !

I am developping a small library that integrates Morphia (MongoDB) and
Elasticsearch using the ES java API.
During application startup I use Annotated Entities to generate the ES
index Mapping.

Everything works pretty fine and I am able to create the index, then
analyzers and the type mappings properly . Indexing works great too.

However I have noted that when restarting the application and going
through the index mapping generation process again (using putMapping
API) the content of the index gets deleted.

I do perform a check to see if the index already exists to avoid
recreating it, however, in my Library the mappings are automatically
uploaded each time the app starts which maybe is the problem ...

Is it a normal behavior of ES? Is the content wiped away when type
mappings are redefined, even if they are exactly the same as the
prevoius ones?
Is there a way to get existing mappings and compare them to the ones
I'm about to upload to avoid this problem?

Hope I'm clear enough

Thanks for your help

Cheers !

Alexandre

Karussell1 · October 21, 2011, 11:08am

I would recommend to reindex the data

only if you changed or removed a field ... for new fields updating
without reindexing should be fine

Alexandre · October 21, 2011, 12:36pm

Can you track down the issues to some curl or java statements?

Sure !

Create index is called at framework startup :

gist.github.com

https://gist.github.com/anonymous/1303673

gistfile1.java


//CREATE INDEX is called the beginning of framework startup

private void createIndex(){
		
		System.out.println("ElasticStore : preparing index " + globalIndexName);
		
		boolean exists = getClient().admin().indices().prepareExists(globalIndexName).execute().actionGet().exists();
		if( !exists){

This file has been truncated. show original

then each entity is analyzed and a mapping is generated using an
XContentBuilder. Here is an example of a generated mapping :

gist.github.com

https://gist.github.com/anonymous/1303707

gistfile1.json

{
	"somethingclass" : {
		"dynamic" : false,
		"properties" : {
			"someProp1" : {
				"index_name" : "monField1",
				"type" : "string",
				"store" : "no",
				"index" : "analyzed",
				"term_vector" : "no",

This file has been truncated. show original

and here are the instructions used to put mapping in es :

gist.github.com

https://gist.github.com/anonymous/1303712

gistfile1.java

public void putMapping(String typeName, XContentBuilder mappingBuilder){
		
		System.out.println(" ----- > putMapping ");
		try {
			
			System.out.println(" >> taking index down --> " + this.globalIndexName);
			getClient().admin().indices().prepareClose(globalIndexName).execute().actionGet();
			
			System.out.println(" >> sending PUT MAPPING : " );
			System.out.println(mappingBuilder.string());

This file has been truncated. show original

Finally insertion of random data .

gist.github.com

https://gist.github.com/anonymous/1303716

gistfile1.txt

private void generate(){
		try {
			
	
		System.out.println("generating");
		 SecureRandom random = new SecureRandom();
		for(int i=0;i<1000;i++){
			
			Random r = new Random();
			int words = r.nextInt(40);

This file has been truncated. show original

Works great on first launch

BUT, if I relaunch it without indexing random stuff ... my previous
indexed content has dissapeard from ES ...

Thanks again for you help

kimchy · October 21, 2011, 6:25pm

Thanks for the gists, one way to simulate it faster is if you have a simpel
"main" program that uses the APIs to shows it. Can you do it?

On Fri, Oct 21, 2011 at 2:36 PM, Alexandre azlist1@gmail.com wrote:

Can you track down the issues to some curl or java statements?

Sure !

Create index is called at framework startup :

gist:1303673 · GitHub

then each entity is analyzed and a mapping is generated using an
XContentBuilder. Here is an example of a generated mapping :

gist:1303707 · GitHub

and here are the instructions used to put mapping in es :

gist:1303712 · GitHub

Finally insertion of random data .

gist:1303716 · GitHub

Works great on first launch

BUT, if I relaunch it without indexing random stuff ... my previous
indexed content has dissapeard from ES ...

Thanks again for you help

Alexandre · October 21, 2011, 10:27pm

Hey !

Thanks for following up !

Here's a full test that you can run.

gist.github.com

https://gist.github.com/anonymous/1305141

gistfile1.java

package test;

import java.io.IOException;
import java.math.BigInteger;
import java.security.SecureRandom;
import java.util.HashMap;
import java.util.Map;
import java.util.Random;

import org.elasticsearch.ElasticSearchException;

This file has been truncated. show original

You will jsut need a small lorem ipsum lib to generate the random
content and its available here :

http://loremipsum.sourceforge.net/

As you will see content is wiped out of the index ... -_-

Let me know what you think.

Regards,

Alexandre

kimchy · October 22, 2011, 1:03am

Hey,

I took your code and ran it (on 0.17.9, and master), and it works for me.
Started both a 1 node cluster, and a 2 nodes cluster, and getting the
correct doc count on both cases. Any more data on your usage?

Btw, few notes on elasticsearch usage. You don't have to close an index to
put the mappings. Also, you can provide the mappings on index creation time
(so you remove the need to have the put mapping call). Last, if you want to
make things searchable "right away", you can simply call the refresh API.

On Sat, Oct 22, 2011 at 12:27 AM, Alexandre azlist1@gmail.com wrote:

Hey !

Thanks for following up !

Here's a full test that you can run.

gist:1305141 · GitHub

You will jsut need a small lorem ipsum lib to generate the random
content and its available here :

http://loremipsum.sourceforge.net/

As you will see content is wiped out of the index ... -_-

Let me know what you think.

Regards,

Alexandre

Alexandre · October 22, 2011, 12:13pm

Hi,

I just upgraded to 0.17.9 and re-ran the test I gave you... and the
problem was still here for me ...

Then I also modified my program a bit to take some of your comments
into account such as using refresh instead of sleep and NOT taking the
index down before putMapping.

And swoosh!! problem seems to have disappeared ...

I don't know what the source of the problem was but maybe there is a
hidden bug to investigate ...

Anyway thank you very much for your help.
I'll probably be bugging you again soon ...

Have a nice day !

kimchy · October 22, 2011, 8:42pm

Heya, it still should not happen. Can you give more details on your
deployment? How big is the cluster (how many nodes), configuration of it.

On Sat, Oct 22, 2011 at 2:13 PM, Alexandre azlist1@gmail.com wrote:

Hi,

I just upgraded to 0.17.9 and re-ran the test I gave you... and the
problem was still here for me ...

Then I also modified my program a bit to take some of your comments
into account such as using refresh instead of sleep and NOT taking the
index down before putMapping.

And swoosh!! problem seems to have disappeared ...

I don't know what the source of the problem was but maybe there is a
hidden bug to investigate ...

Anyway thank you very much for your help.
I'll probably be bugging you again soon ...

Have a nice day !

Alexandre · October 22, 2011, 9:41pm

This is a dev / test server so I only have 1 node + 1 shard + 0
replicas

1 client connecting to it on the same machine.

Alexandre · October 22, 2011, 9:42pm

Configuration is the default config I think ... I didnt change
anything as far as I can remember.

kimchy · October 23, 2011, 12:12am

tried to recreate it again with a single node, 1 shard 0 replica index, and
still seems to work.. . Maybe you can check what config you used?

On Sat, Oct 22, 2011 at 11:42 PM, Alexandre azlist1@gmail.com wrote:

Configuration is the default config I think ... I didnt change
anything as far as I can remember.

Alexandre · October 23, 2011, 12:48am

Ok so first of all here are all the configuration infos I can find in
my GUI :

gist.github.com

https://gist.github.com/anonymous/1306691

cluster_health.json

//CLUSTER HEALTH

{

    cluster_name: elasticsearch
    status: green
    timed_out: false
    number_of_nodes: 1
    number_of_data_nodes: 1
    active_primary_shards: 1

This file has been truncated. show original

cluster_nodes.json

//CLUSTER NODES
{

    cluster_name: elasticsearch
    nodes: {
        3uKiF-aqRtmYcK3vEBm8Sw: {
            name: X-23
            transport_address: inet[/192.168.0.52:9300]
            attributes: { }
            http_address: inet[/192.168.0.52:9200]

This file has been truncated. show original

cluster_state.json

//CLUSTER STATE
{

    cluster_name: elasticsearch
    master_node: 3uKiF-aqRtmYcK3vEBm8Sw
    blocks: { }
    nodes: {
        3uKiF-aqRtmYcK3vEBm8Sw: {
            name: X-23
            transport_address: inet[/192.168.0.52:9300]

This file has been truncated. show original

There are more than three files. show original

Next I noticed something strange in my console that occurs when
triggering the createIndex(true) method of the test I gave you.
I don0t if its related to the same problem ...

In the logs I get the following :

Exception caught on netty layer [[id: 0x552297d7, /192.168.0.52:55124
=> /192.168.0.52:9301]]

here is the full log of what happened just when creating the index.

gist.github.com

https://gist.github.com/anonymous/1306700

gistfile1.txt

[2011-10-23 02:30:16,029][INFO ][node                     ] [X-23] {elasticsearch/0.17.9}[8500]: initializing ...
[2011-10-23 02:30:16,033][INFO ][plugins                  ] [X-23] loaded [], sites []
[2011-10-23 02:30:17,462][INFO ][node                     ] [X-23] {elasticsearch/0.17.9}[8500]: initialized
[2011-10-23 02:30:17,463][INFO ][node                     ] [X-23] {elasticsearch/0.17.9}[8500]: starting ...
[2011-10-23 02:30:17,544][INFO ][transport                ] [X-23] bound_address {inet[/0.0.0.0:9300]}, publish_address {inet[/192.168.0.52:9300]}
[2011-10-23 02:30:20,615][INFO ][cluster.service          ] [X-23] new_master [X-23][3uKiF-aqRtmYcK3vEBm8Sw][inet[/192.168.0.52:9300]], reason: zen-disco-join (elected_as_master)
[2011-10-23 02:30:20,690][INFO ][discovery                ] [X-23] elasticsearch/3uKiF-aqRtmYcK3vEBm8Sw
[2011-10-23 02:30:20,704][INFO ][gateway                  ] [X-23] recovered [0] indices into cluster_state
[2011-10-23 02:30:20,730][INFO ][http                     ] [X-23] bound_address {inet[/0.0.0.0:9200]}, publish_address {inet[/192.168.0.52:9200]}
[2011-10-23 02:30:20,731][INFO ][node                     ] [X-23] {elasticsearch/0.17.9}[8500]: started

This file has been truncated. show original

Alexandre · October 23, 2011, 12:59am

Plus I am now sure that the index wiping problem ONLY occurs when the
index is taken down before putMapping calls in the test ...

If I comment out the close + open instructions the problem does not
occur ...

//
getClient().admin().indices().prepareClose(globalIndexname).execute().actionGet();

//
getClient().admin().indices().prepareOpen(globalIndexname).execute().actionGet();

Alexandre · October 24, 2011, 11:38am

Hey !

Just to let you know :

I just tested this again today on a different machine (32bit vista)
with a fresh install of ES 0.17.9 with no configuration changes from
what was downloaded and the weird index wiping still occurred.

When the index is taken down before putMapping (even though its not
necessary), the first putMapping call works fine and documents are
indexed but the seccond putMapping call (with same mapping) wipes
the index.

When the index is NOT taken down both putMapping call work fine and
the documents are still there after both calls.

Hope this helps you.

Cheers !

Alex