Kafka looks interesting, though at this point we're actively trying to
reduce the number of moving parts, so I think an AMQ based approach is what
we'll ultimately go for.
Seems like there might be room here for an
elasticsearch-elasticsearch-river plugin or something - to do one or two
way close to real time replication on some selected set of indexes between
separate clusters. That way you could easily mirror prod data to a dev
environment without depending on the ability to do the duplication earlier
in the pipeline, or depending on scripts to move the data around.
On Wednesday, March 12, 2014 2:46:19 PM UTC-7, Otis Gospodnetic wrote:
Consider Kafka 0.8.1. It comes with a MirrorMaker tool that mirrors Kafka
data (to multiple DCs). Once data is local, you can feed your ES from the
local Kafka broker.
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
On Wednesday, March 12, 2014 2:55:58 PM UTC-4, Josh Harrison wrote:
Say I have clusters A and B. Cluster A is consuming data using an
ActiveMQ river. I would like to stream data to cluster B as well. Do I just
create a secondary outbound AMQ channel and subscribe cluster B to it, or
is there a decent way to have a live copy of data going two places at once?
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firstname.lastname@example.org.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6b1bdbe4-e2fa-4b10-9298-62d3d1869842%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.