# Strange behaviour on cluster of two nodes on aws **URL:** \ **Category:** Elasticsearch\ **Created:** [November 24, 2012, 7:18am UTC](https://discuss.elastic.co/t/strange-behaviour-on-cluster-of-two-nodes-on-aws/9810 "2012-11-24T07:18:26Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Tarun\_Jangra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarun_jangra/32/1426_2.png) [@Tarun\_Jangra](https://discuss.elastic.co/u/Tarun_Jangra)\ **Post date:** [November 24, 2012, 7:18am UTC](https://discuss.elastic.co/t/strange-behaviour-on-cluster-of-two-nodes-on-aws/9810/1 "2012-11-24T07:18:26Z")
Hi, We have cluster of two machines and they are recognising each other perfectly. Where one is primary node and other is working as a replica. Health check is also Green. Task scenario is. 1. Pushing new document on primary. 2. after pushing, same thread is calculating one more document and try to add one more metadata using update IP in the same document. 3. Getting Mapping parser exception. I investigated further, and found while pushing this new metadata in the document, it is not finding same document. Well than i put a sleep of 1 sec between (1 and 2) step. And it starts working. I doubt, since i added replica and may be because of network latency it is taking time to get publish the main document and script is trying to add metadata for the document which is not existing yet. i also tried refresh API between setup 1 and 2. But it did not work. one more strage thing. Some times it starts working sometimes is starts generating mapper parser exception. I believe i am missing something. Can someone put me in right direction to get this fix. Thank you. ---
**Author:** ![radu\_gheorghe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radu_gheorghe/32/556_2.png) [@radu\_gheorghe](https://discuss.elastic.co/u/radu_gheorghe)\ **Post date:** [November 24, 2012, 9:44am UTC](https://discuss.elastic.co/t/strange-behaviour-on-cluster-of-two-nodes-on-aws/9810/2 "2012-11-24T09:44:24Z")
Hello Terry, Sorry if I point out the obvious, but I think it would be a good idea to gather all the document information and insert everything at once. ## Best regards, Radu [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene On Sat, Nov 24, 2012 at 9:18 AM, Terry [tarun@izap.in](mailto:tarun@izap.in) wrote: > Hi, > > We have cluster of two machines and they are recognising each other > perfectly. Where one is primary node and other is working as a replica. > Health check is also Green. > > Task scenario is. > > 1. Pushing new document on primary. > 2. after pushing, same thread is calculating one more document and try to > add one more metadata using update IP in the same document. > 3. Getting Mapping parser exception. > > I investigated further, and found while pushing this new metadata in the > document, it is not finding same document. Well than i put a sleep of 1 sec > between (1 and 2) step. And it starts working. > > I doubt, since i added replica and may be because of network latency it is > taking time to get publish the main document and script is trying to add > metadata for the document which is not existing yet. > > i also tried refresh API between setup 1 and 2. But it did not work. > > one more strage thing. Some times it starts working sometimes is starts > generating mapper parser exception. > > I believe i am missing something. Can someone put me in right direction to > get this fix. > > Thank you. > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Strange-behaviour-on-cluster-of-two-nodes-on-aws-tp4025951.html](http://elasticsearch-users.115913.n3.nabble.com/Strange-behaviour-on-cluster-of-two-nodes-on-aws-tp4025951.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). > > -- -- ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 24, 2012, 9:45am UTC](https://discuss.elastic.co/t/strange-behaviour-on-cluster-of-two-nodes-on-aws/9810/3 "2012-11-24T09:45:34Z")
Hi Terry, It looks strange. Could you gist a curl recreation? It will help us to reproduce your issue. IMHO it should work, especialy if you have added the refresh Call. That said, perhaps there is a latency that put your cluster in Red Health just after the first doc insertion. Did you try to create your index first? Then wait for Yellow status and then send your first doc and its update? -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 24 nov. 2012 à 08:18, Terry [tarun@izap.in](mailto:tarun@izap.in) a écrit : Hi, We have cluster of two machines and they are recognising each other perfectly. Where one is primary node and other is working as a replica. Health check is also Green. Task scenario is. 1. Pushing new document on primary. 2. after pushing, same thread is calculating one more document and try to add one more metadata using update IP in the same document. 3. Getting Mapping parser exception. I investigated further, and found while pushing this new metadata in the document, it is not finding same document. Well than i put a sleep of 1 sec between (1 and 2) step. And it starts working. I doubt, since i added replica and may be because of network latency it is taking time to get publish the main document and script is trying to add metadata for the document which is not existing yet. i also tried refresh API between setup 1 and 2. But it did not work. one more strage thing. Some times it starts working sometimes is starts generating mapper parser exception. I believe i am missing something. Can someone put me in right direction to get this fix. Thank you. -- View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Strange-behaviour-on-cluster-of-two-nodes-on-aws-tp4025951.html](http://elasticsearch-users.115913.n3.nabble.com/Strange-behaviour-on-cluster-of-two-nodes-on-aws-tp4025951.html) Sent from the ElasticSearch Users mailing list archive at [Nabble.com](http://Nabble.com). -- -- ---
**Author:** ![Tarun\_Jangra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarun_jangra/32/1426_2.png) [@Tarun\_Jangra](https://discuss.elastic.co/u/Tarun_Jangra)\ **Post date:** [December 23, 2012, 6:11pm UTC](https://discuss.elastic.co/t/strange-behaviour-on-cluster-of-two-nodes-on-aws/9810/4 "2012-12-23T18:11:26Z")
Hi Guys, Thank you for your reply. Actually it was hurry to deliver a product. So i made something to get this done. but i would like to share what i observe. Because i still need the solution of this problem. EC2 Layout. 1 primary Es Server (nginx + elasticsearch) 1 Replica Es Server (nginx + elasticsearch) i want to restrict elasticsearch access by username, password. so that's why i used Nginx which was supposed to be proxy for Es access. Since php was interfacing elasticsearch via Nginx. And somehow it was creating this problem. for the timebeing, i stopped using nginx and directly using elasticsearch. It is working fine. Is there any clue i could manage to get this work? Thank you. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 23, 2012, 7:20pm UTC](https://discuss.elastic.co/t/strange-behaviour-on-cluster-of-two-nodes-on-aws/9810/5 "2012-12-23T19:20:52Z")
So you have probably misconfigured your nginx instances. Why do you have two nginx instances? Why do you call one of your es as a primary and the other one as secondary? Can't you build a cluster of Es nodes and a single Nginx on top of them? Each ES must be able to talk with each other on 9300-9399. Not sure it helps, but I'm pretty sure that you should double-check your nginx settings. ## HTH David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 23 déc. 2012 à 19:11, Terry [tarun@izap.in](mailto:tarun@izap.in) a écrit : Hi Guys, Thank you for your reply. Actually it was hurry to deliver a product. So i made something to get this done. but i would like to share what i observe. Because i still need the solution of this problem. EC2 Layout. 1 primary Es Server (nginx + elasticsearch) 1 Replica Es Server (nginx + elasticsearch) i want to restrict elasticsearch access by username, password. so that's why i used Nginx which was supposed to be proxy for Es access. Since php was interfacing elasticsearch via Nginx. And somehow it was creating this problem. for the timebeing, i stopped using nginx and directly using elasticsearch. It is working fine. Is there any clue i could manage to get this work? Thank you. -- View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Strange-behaviour-on-cluster-of-two-nodes-on-aws-tp4025951p4027376.html](http://elasticsearch-users.115913.n3.nabble.com/Strange-behaviour-on-cluster-of-two-nodes-on-aws-tp4025951p4027376.html) Sent from the ElasticSearch Users mailing list archive at [Nabble.com](http://Nabble.com). -- -- ---
**Author:** ![Tarun\_Jangra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tarun_jangra/32/1426_2.png) [@Tarun\_Jangra](https://discuss.elastic.co/u/Tarun_Jangra)\ **Post date:** [December 27, 2012, 8:06pm UTC](https://discuss.elastic.co/t/strange-behaviour-on-cluster-of-two-nodes-on-aws/9810/6 "2012-12-27T20:06:01Z")
Thank you David, Yes, you are right. I can do that as long as we have two servers. one primary and one replica. But we would add more servers in future. And we do need to implement "round ribbon" in future. where our script is suppose to communicate directly with every elasticsearch node. It will help us to distribute the query load on multiple servers in a single thread. So to have password protected access. we do need to put nigix on every server. Not sure if there is any better solution. Thank you. ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [December 31, 2012, 1:06pm UTC](https://discuss.elastic.co/t/strange-behaviour-on-cluster-of-two-nodes-on-aws/9810/7 "2012-12-31T13:06:54Z")
It would be great if you could provide a repro for this issue. I am not quite sure why you are calling one server "primary" and another one "replica" since "primary" and "replica" status is assign on per shard basis. So, unless you have one shard per service, "primary" and "replica" are not really applicable on the server level. By the way, why does your script need to communicate directly with every elasticsearch node? Igor On Thursday, December 27, 2012 3:06:01 PM UTC-5, Rimpy wrote: > Thank you David, Yes, you are right. I can do that as long as we have two > servers. one primary and one replica. > > But we would add more servers in future. And we do need to implement > "round > ribbon" in future. where our script is suppose to communicate directly > with > every elasticsearch node. It will help us to distribute the query load on > multiple servers in a single thread. > > So to have password protected access. we do need to put nigix on every > server. Not sure if there is any better solution. > > Thank you. > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Strange-behaviour-on-cluster-of-two-nodes-on-aws-tp4025951p4027492.html](http://elasticsearch-users.115913.n3.nabble.com/Strange-behaviour-on-cluster-of-two-nodes-on-aws-tp4025951p4027492.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:58am UTC](https://discuss.elastic.co/t/strange-behaviour-on-cluster-of-two-nodes-on-aws/9810/8 "2017-07-06T02:58:09Z")