Hi all,
We have a deployment scenario I can't seem to find any examples of, and
any help would be greatly appreciated. We're running ElasticSearch in 3
AWS regions. We want these regions to survive a failure from other
regions, and we want all writes and reads from our clients to occur in the
local region. Rather than have 1 large cluster that spans 3 regions, I
would like the ability to asynchronously replicate document creation and
deletion across the regions. Our application creates immutable documents,
so update semantics don't apply. Is there any way this can be accomplished
currently? Note that I don't think a river will provide us with the
partition we need since it is a singleton.
Hi all,
We have a deployment scenario I can't seem to find any examples of, and
any help would be greatly appreciated. We're running Elasticsearch in 3
AWS regions. We want these regions to survive a failure from other
regions, and we want all writes and reads from our clients to occur in the
local region. Rather than have 1 large cluster that spans 3 regions, I
would like the ability to asynchronously replicate document creation and
deletion across the regions. Our application creates immutable documents,
so update semantics don't apply. Is there any way this can be accomplished
currently? Note that I don't think a river will provide us with the
partition we need since it is a singleton.
Hey Mark,
We're looking for a solution that's faster than a snapshot+restore. We
do follow an eventual consistency model, but we'd like the replication to
be near realtime, so a snapshot and restore would be too slow. We really
need something that synchronizes between regions with some sort of log or
queue, so that if a region goes down, we can replay all missed operations.
We were going to put an SQS queue in our application tier, but if this is
something an ES plugin can provide, we would prefer to utilize that over
writing our own.
I was searching the plugin api, but I can't seem to find anything that will
receive post document write/delete events I can hook into. Ideally, I
would like each node to queue up changes to primary shards. This way, we
can distribute sending the changes among all our nodes and we won't have a
single point of failure.
Hi all,
We have a deployment scenario I can't seem to find any examples of, and
any help would be greatly appreciated. We're running Elasticsearch in 3
AWS regions. We want these regions to survive a failure from other
regions, and we want all writes and reads from our clients to occur in the
local region. Rather than have 1 large cluster that spans 3 regions, I
would like the ability to asynchronously replicate document creation and
deletion across the regions. Our application creates immutable documents,
so update semantics don't apply. Is there any way this can be accomplished
currently? Note that I don't think a river will provide us with the
partition we need since it is a singleton.
Hey Mark,
We're looking for a solution that's faster than a snapshot+restore. We
do follow an eventual consistency model, but we'd like the replication to
be near realtime, so a snapshot and restore would be too slow. We really
need something that synchronizes between regions with some sort of log or
queue, so that if a region goes down, we can replay all missed operations.
We were going to put an SQS queue in our application tier, but if this is
something an ES plugin can provide, we would prefer to utilize that over
writing our own.
I was searching the plugin api, but I can't seem to find anything that
will receive post document write/delete events I can hook into. Ideally, I
would like each node to queue up changes to primary shards. This way, we
can distribute sending the changes among all our nodes and we won't have a
single point of failure.
Hi all,
We have a deployment scenario I can't seem to find any examples of,
and any help would be greatly appreciated. We're running Elasticsearch in
3 AWS regions. We want these regions to survive a failure from other
regions, and we want all writes and reads from our clients to occur in the
local region. Rather than have 1 large cluster that spans 3 regions, I
would like the ability to asynchronously replicate document creation and
deletion across the regions. Our application creates immutable documents,
so update semantics don't apply. Is there any way this can be accomplished
currently? Note that I don't think a river will provide us with the
partition we need since it is a singleton.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.