Topology of tribe nodes in a distributed replicated environment and automatic index + alias creation

Hey guys,
We have a slightly different use case than I'm able to find examples for
with the tribe node. Any feedback would be appreciated.

What we have now:

Single region:

We create indexes in our code automatically. They're based on timeuuids,
so we never have to worry about them conflicting. We also create read and
write aliases to these indexes.
We read and write documents to our local region only.

What we want:

2+ Region Topology

us-west, us-east, asia pac

What I'm envisioning.

Our Code: We implement a tool that will create a log of all index
creations, as well as read and write alias creation. These operations are
then replayed individually on each region's cluster. If a region is
offline, or unreachable, it won't block other regions getting created.

Tribe Nodes: Write to all regions. Is there a way we can do this so that
this returns to the caller after our local region has accepted the write?
We don't want to wait for all regions to ack the write. Again if a region
is offline, we'll lose that ability. We can re-index those documents later.

Read: Always read from local cluster for efficiency and responsiveness.

We're exploring all options for multi region deployments. Ultimately we'd
like reads to occur locally for performance, and are ok with queueing
background writes to each region.

Thoughts?

Thanks,
Todd

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e6dcf03c-b023-4118-ac9b-4cf6701e4885%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

You need to be careful here because if you are using the same alias for a
given in index on all your cluster then the tribe node won't know where it
needs to go.
But your queuing strategy sounds good.

Regarding querying from a local cluster, you can use
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-preference.html

On 12 February 2015 at 13:32, Todd Nine tnine@apigee.com wrote:

Hey guys,
We have a slightly different use case than I'm able to find examples for
with the tribe node. Any feedback would be appreciated.

What we have now:

Single region:

We create indexes in our code automatically. They're based on timeuuids,
so we never have to worry about them conflicting. We also create read and
write aliases to these indexes.
We read and write documents to our local region only.

What we want:

2+ Region Topology

us-west, us-east, asia pac

What I'm envisioning.

Our Code: We implement a tool that will create a log of all index
creations, as well as read and write alias creation. These operations are
then replayed individually on each region's cluster. If a region is
offline, or unreachable, it won't block other regions getting created.

Tribe Nodes: Write to all regions. Is there a way we can do this so that
this returns to the caller after our local region has accepted the write?
We don't want to wait for all regions to ack the write. Again if a region
is offline, we'll lose that ability. We can re-index those documents later.

Read: Always read from local cluster for efficiency and responsiveness.

We're exploring all options for multi region deployments. Ultimately we'd
like reads to occur locally for performance, and are ok with queueing
background writes to each region.

Thoughts?

Thanks,
Todd

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e6dcf03c-b023-4118-ac9b-4cf6701e4885%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e6dcf03c-b023-4118-ac9b-4cf6701e4885%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X88Ty5dLvBcFfN8x2O9f-RAyBR2iPViD92FNcnfKBWTHQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.