Kafka looks interesting, though at this point we're actively trying to
reduce the number of moving parts, so I think an AMQ based approach is what
we'll ultimately go for.
Seems like there might be room here for an
elasticsearch-elasticsearch-river plugin or something - to do one or two
way close to real time replication on some selected set of indexes between
separate clusters. That way you could easily mirror prod data to a dev
environment without depending on the ability to do the duplication earlier
in the pipeline, or depending on scripts to move the data around.
On Wednesday, March 12, 2014 2:46:19 PM UTC-7, Otis Gospodnetic wrote:
Consider Kafka 0.8.1. It comes with a MirrorMaker tool that mirrors Kafka
data (to multiple DCs). Once data is local, you can feed your ES from the
local Kafka broker.Otis
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/On Wednesday, March 12, 2014 2:55:58 PM UTC-4, Josh Harrison wrote:
Say I have clusters A and B. Cluster A is consuming data using an
ActiveMQ river. I would like to stream data to cluster B as well. Do I just
create a secondary outbound AMQ channel and subscribe cluster B to it, or
is there a decent way to have a live copy of data going two places at once?
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6b1bdbe4-e2fa-4b10-9298-62d3d1869842%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.