I've developed an elasticsearch plugin which wathces the index changes and
broadcasts them to a websocket running within the plugin. And ofcourse I
ran into a problem If i use a cluster of 3 (for example) elasticsearch
services, writing a new document won't trigger the index change event on
all es servers but only on those having the shard. My questing is how can I
force es to write all the shards to all the servers at all times but at the
same time keep the scaleability? By keeping the scaleability I mean an
option to add more servers without the need of changing the config files
with number of replicas and restarting the services.
I've tried playing with write_consistency but I believe that's totally
different thing.
I've developed an elasticsearch plugin which wathces the index changes and
broadcasts them to a websocket running within the plugin. And ofcourse I
ran into a problem If i use a cluster of 3 (for example) elasticsearch
services, writing a new document won't trigger the index change event on
all es servers but only on those having the shard. My questing is how can I
force es to write all the shards to all the servers at all times but at the
same time keep the scaleability? By keeping the scaleability I mean an
option to add more servers without the need of changing the config files
with number of replicas and restarting the services.
I've tried playing with write_consistency but I believe that's totally
different thing.
My idea is to have a cluster of es with a plugin and an nginx above it to
balance the sockets. In this case each plugin has to broadcast all changes
made in the es,
for this purpose I need to have all the shards on all the servers, or am I
wrong?
On Friday, April 4, 2014 3:28:21 PM UTC+4, Jörg Prante wrote:
In my websocket plugin, I solve this by a pub/sub architecture
each node with websocket plugin opens a channel to every other node with
websocket plugin
a publisher offers a service to which other clients can subscribe
the result is a service where a number of subscribing clients are
permanently connected and wait for messages
if a client pushes a document, he can also publish a success message to
the service
every subscribing client is notified about the message (implemented as
scan/scroll action over a dedicated pub/sub index)
On Fri, Apr 4, 2014 at 9:39 AM, Simon Even <even....@gmail.com<javascript:>
wrote:
Good day!
I've developed an elasticsearch plugin which wathces the index changes
and broadcasts them to a websocket running within the plugin. And ofcourse
I ran into a problem If i use a cluster of 3 (for example) elasticsearch
services, writing a new document won't trigger the index change event on
all es servers but only on those having the shard. My questing is how can I
force es to write all the shards to all the servers at all times but at the
same time keep the scaleability? By keeping the scaleability I mean an
option to add more servers without the need of changing the config files
with number of replicas and restarting the services.
I've tried playing with write_consistency but I believe that's totally
different thing.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
If i use a cluster of 3 (for example) elasticsearch
If i use a cluster of 3 (for example) elasticsearch