We have cluster of two machines and they are recognising each other perfectly. Where one is primary node and other is working as a replica. Health check is also Green.
Task scenario is.
Pushing new document on primary.
after pushing, same thread is calculating one more document and try to add one more metadata using update IP in the same document.
Getting Mapping parser exception.
I investigated further, and found while pushing this new metadata in the document, it is not finding same document. Well than i put a sleep of 1 sec between (1 and 2) step. And it starts working.
I doubt, since i added replica and may be because of network latency it is taking time to get publish the main document and script is trying to add metadata for the document which is not existing yet.
i also tried refresh API between setup 1 and 2. But it did not work.
one more strage thing. Some times it starts working sometimes is starts generating mapper parser exception.
I believe i am missing something. Can someone put me in right direction to get this fix.
On Sat, Nov 24, 2012 at 9:18 AM, Terry tarun@izap.in wrote:
Hi,
We have cluster of two machines and they are recognising each other
perfectly. Where one is primary node and other is working as a replica.
Health check is also Green.
Task scenario is.
Pushing new document on primary.
after pushing, same thread is calculating one more document and try to
add one more metadata using update IP in the same document.
Getting Mapping parser exception.
I investigated further, and found while pushing this new metadata in the
document, it is not finding same document. Well than i put a sleep of 1 sec
between (1 and 2) step. And it starts working.
I doubt, since i added replica and may be because of network latency it is
taking time to get publish the main document and script is trying to add
metadata for the document which is not existing yet.
i also tried refresh API between setup 1 and 2. But it did not work.
one more strage thing. Some times it starts working sometimes is starts
generating mapper parser exception.
I believe i am missing something. Can someone put me in right direction to
get this fix.
It looks strange. Could you gist a curl recreation?
It will help us to reproduce your issue.
IMHO it should work, especialy if you have added the refresh Call.
That said, perhaps there is a latency that put your cluster in Red Health just after the first doc insertion.
Did you try to create your index first? Then wait for Yellow status and then send your first doc and its update?
--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 24 nov. 2012 à 08:18, Terry tarun@izap.in a écrit :
Hi,
We have cluster of two machines and they are recognising each other
perfectly. Where one is primary node and other is working as a replica.
Health check is also Green.
Task scenario is.
Pushing new document on primary.
after pushing, same thread is calculating one more document and try to
add one more metadata using update IP in the same document.
Getting Mapping parser exception.
I investigated further, and found while pushing this new metadata in the
document, it is not finding same document. Well than i put a sleep of 1 sec
between (1 and 2) step. And it starts working.
I doubt, since i added replica and may be because of network latency it is
taking time to get publish the main document and script is trying to add
metadata for the document which is not existing yet.
i also tried refresh API between setup 1 and 2. But it did not work.
one more strage thing. Some times it starts working sometimes is starts
generating mapper parser exception.
I believe i am missing something. Can someone put me in right direction to
get this fix.
Thank you for your reply. Actually it was hurry to deliver a product. So i made something to get this done. but i would like to share what i observe. Because i still need the solution of this problem.
EC2 Layout.
1 primary Es Server (nginx + elasticsearch)
1 Replica Es Server (nginx + elasticsearch)
i want to restrict elasticsearch access by username, password. so that's why i used Nginx which was supposed to be proxy for Es access.
Since php was interfacing elasticsearch via Nginx. And somehow it was creating this problem. for the timebeing, i stopped using nginx and directly using elasticsearch. It is working fine.
Is there any clue i could manage to get this work?
So you have probably misconfigured your nginx instances.
Why do you have two nginx instances? Why do you call one of your es as a primary and the other one as secondary?
Can't you build a cluster of Es nodes and a single Nginx on top of them?
Each ES must be able to talk with each other on 9300-9399.
Not sure it helps, but I'm pretty sure that you should double-check your nginx settings.
HTH
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 23 déc. 2012 à 19:11, Terry tarun@izap.in a écrit :
Hi Guys,
Thank you for your reply. Actually it was hurry to deliver a product. So i
made something to get this done. but i would like to share what i observe.
Because i still need the solution of this problem.
EC2 Layout.
1 primary Es Server (nginx + elasticsearch)
1 Replica Es Server (nginx + elasticsearch)
i want to restrict elasticsearch access by username, password. so that's why
i used Nginx which was supposed to be proxy for Es access.
Since php was interfacing elasticsearch via Nginx. And somehow it was
creating this problem. for the timebeing, i stopped using nginx and directly
using elasticsearch. It is working fine.
Is there any clue i could manage to get this work?
Thank you David, Yes, you are right. I can do that as long as we have two servers. one primary and one replica.
But we would add more servers in future. And we do need to implement "round ribbon" in future. where our script is suppose to communicate directly with every elasticsearch node. It will help us to distribute the query load on multiple servers in a single thread.
So to have password protected access. we do need to put nigix on every server. Not sure if there is any better solution.
It would be great if you could provide a repro for this issue.
I am not quite sure why you are calling one server "primary" and another
one "replica" since "primary" and "replica" status is assign on per shard
basis. So, unless you have one shard per service, "primary" and "replica"
are not really applicable on the server level. By the way, why does your
script need to communicate directly with every elasticsearch node?
Igor
On Thursday, December 27, 2012 3:06:01 PM UTC-5, Rimpy wrote:
Thank you David, Yes, you are right. I can do that as long as we have two
servers. one primary and one replica.
But we would add more servers in future. And we do need to implement
"round
ribbon" in future. where our script is suppose to communicate directly
with
every elasticsearch node. It will help us to distribute the query load on
multiple servers in a single thread.
So to have password protected access. we do need to put nigix on every
server. Not sure if there is any better solution.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
