# We are losing data in elastic search cluster. v2.4.0 **URL:** \ **Category:** Elasticsearch\ **Created:** [October 2, 2016, 6:52pm UTC](https://discuss.elastic.co/t/we-are-losing-data-in-elastic-search-cluster-v2-4-0/62011 "2016-10-02T18:52:11Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Mustafa\_Halil\_YILDIZ](https://avatars.discourse-cdn.com/v4/letter/m/3bc359/32.png) [@Mustafa\_Halil\_YILDIZ](https://discuss.elastic.co/u/Mustafa_Halil_YILDIZ)\ **Post date:** [October 2, 2016, 6:52pm UTC](https://discuss.elastic.co/t/we-are-losing-data-in-elastic-search-cluster-v2-4-0/62011/1 "2016-10-02T18:52:11Z")
We made a poc with ElasticSearch but while doing it, we lost data in clustered enviroment. We used ES 2.4.0. Can anyone say what we are missing? Our scenario is: 1- Open Elastic Server-1 and Server-2 with the configurations below, they are in a cluster. Our Configurations: ` ***Server-1**** cluster.name: ESCluster node.master: true node.name: "es1" node.data: true network.bind_host: ["127.0.0.1","20.20.20.5"] network.publish_host: "20.20.20.5" discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["20.20.20.5","20.20.20.6"] discovery.zen.minimum_master_nodes: 1` ` ***Server-2**** cluster.name: ESCluster node.master: true node.name: "es2" node.data: true network.bind_host: ["127.0.0.1","20.20.20.6"] network.publish_host: "20.20.20.6" discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["20.20.20.5","20.20.20.6"] discovery.zen.minimum_master_nodes: 1` 2- Index document over Server-1: `curl -XPUT '20.20.20.5:9200/ert/post/1' -d ' { "user": "easlan", "postDate": "01-16-2015", "body": "Adding Data in ElasticSearch Cluster" , "title": "ElasticSearch Cluster Test - 1" }'` 3- Look for indexed docs over Server-1 or Server-2:Total number of results is 1(as expected): `curl -XGET '20.20.20.5:9200/ert/post/_search?q=user:easlan&pretty=true' curl -XGET '20.20.20.6:9200/ert/post/_search?q=user:easlan&pretty=true'` 4- Then close Server-1 5- Index new document over Server-2: `curl -XPUT '20.20.20.6:9200/ert/post/2' -d ' { "user": "easlan", "postDate": "01-16-2015", "body": "Adding Data in ElasticSearch Cluster" , "title": "ElasticSearch Cluster Test - 2" }'` 6- Look for indexed docs over Server-2:Total number of results is 2: `curl -XGET '20.20.20.6:9200/ert/post/_search?q=user:easlan&pretty=true'` 7- Close Server-2 8- Open Server-1 9- Look for indexed docs over Server-1:Total number of results is 1 (as expected, because server-2 is closed): `curl -XGET '20.20.20.5:9200/ert/post/_search?q=user:easlan&pretty=true'` 10- Then open Server-2 again. Look for indexed docs over Server-1 or Server-2. We expect to see total number of results as 2 but when we look, we got 1 as a result. Even we restart two of them again the result is still 1: `curl -XGET '20.20.20.5:9200/ert/post/_search?q=user:easlan&pretty=true' curl -XGET '20.20.20.6:9200/ert/post/_search?q=user:easlan&pretty=true'` my\_server\_ip:9200/\_nodes/stats result is at [here](https://www.dropbox.com/s/5jby1p6buvld5km/node_stats.json?dl=0) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 2, 2016, 7:18pm UTC](https://discuss.elastic.co/t/we-are-losing-data-in-elastic-search-cluster-v2-4-0/62011/2 "2016-10-02T19:18:58Z")
As you have 2 master eligible nodes, you must set `minimum_master_nodes` to 2, not 1 as in your config. `minimum_master_nodes` should always be set to `ceiling((N/2) + 1)`. This setting will prevent writes as long as both nodes are not available in order to avoid data loss. If you want a cluster that can continue taking writes even with one node down, you need at least 3 master eligible nodes. You can read more about this [here](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/important-configuration-changes.html#_minimum_master_nodes). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 2, 2016, 11:05pm UTC](https://discuss.elastic.co/t/we-are-losing-data-in-elastic-search-cluster-v2-4-0/62011/3 "2016-10-02T23:05:54Z")
Consistency is not magic. As @Christian_Dahlqvist mentions you have bad quorum. At step 10 which node is the master? ---
**Author:** ![Mustafa\_Halil\_YILDIZ](https://avatars.discourse-cdn.com/v4/letter/m/3bc359/32.png) [@Mustafa\_Halil\_YILDIZ](https://discuss.elastic.co/u/Mustafa_Halil_YILDIZ)\ **Post date:** [October 3, 2016, 11:43am UTC](https://discuss.elastic.co/t/we-are-losing-data-in-elastic-search-cluster-v2-4-0/62011/4 "2016-10-03T11:43:45Z")
Hi @Christian_Dahlqvist, thanks for your reply. We understood that we were thinking a little different. This solves our problem. ---
**Author:** ![Mustafa\_Halil\_YILDIZ](https://avatars.discourse-cdn.com/v4/letter/m/3bc359/32.png) [@Mustafa\_Halil\_YILDIZ](https://discuss.elastic.co/u/Mustafa_Halil_YILDIZ)\ **Post date:** [October 3, 2016, 11:47am UTC](https://discuss.elastic.co/t/we-are-losing-data-in-elastic-search-cluster-v2-4-0/62011/5 "2016-10-03T11:47:21Z")
hi @warkolm , server 1 is the master at step 10. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:15pm UTC](https://discuss.elastic.co/t/we-are-losing-data-in-elastic-search-cluster-v2-4-0/62011/6 "2017-07-05T22:15:29Z")