Elasticsearch and Hadoop on the same cluster

Elastic Stack Elasticsearch
1

Greatings everyone, I repost this just in case :

The Elasticseach-hadoop current guide mentions a two clusters
configurations: eslasticsearch-hadoop handles communications between the
Hadoop cluster and the Elasticsearch one.
Link :
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/arch.html

I am wondering about the other obvious configuration: one of shared nodes,
consisting in one cluster for both Hadoop and Elasticsearch. The case is
mentioned in Elasticsearch-Hortonworks' presentation (slides 31-32) :

How would one implement Hadoop and Elasticsearch on the same cluster?
Would there be any trouble between Hadoop ZooKeeper and Elasticsearch's own
cluster management?

Thanks to all for paying attention!

Charles

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5c186974-78b9-4d34-9271-ca32b653c5bb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Hi Charles,

No need to repost - if you are looking for real-time replies, try the IRC.
What exactly are you looking for? Elasticsearch and Hadoop are 'services' that can share the same hardware.
Since ES does not depend on Hadoop or vice-versa, you can install each one as you typically do - the more hardware, the
better.
You basically end up with two 'software' clusters on top of a hardware one. The advantage here is that you can mix and
match:
you can choose to allocate a different number of machines to ES vs Hadoop; some machines can be shared while others
aren't - it's your choice.

There's no "implementation" to be done - you just install them one by one and if you want to run your Hadoop jobs
against ES,
use es-hadoop (the documentation contains plenty of examples for each library that you might use from Map/Reduce to Spark).

On 1/14/15 12:04 PM, Charles Vallée wrote:

Greatings everyone, I repost this just in case :

The Elasticseach-hadoop current guide mentions a two clusters configurations: eslasticsearch-hadoop handles
communications between the Hadoop cluster and the Elasticsearch one.
Link : Elasticsearch Platform — Find real-time answers at scale | Elastic
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/arch.html

I am wondering about the other obvious configuration: one of shared nodes, consisting in one cluster for both Hadoop and
Elasticsearch. The case is mentioned in Elasticsearch-Hortonworks' presentation (slides 31-32) :
Combine Apache Hadoop and Elasticsearch to Get the Most of Your Big Data | PPT
http://fr.slideshare.net/hortonworks/hortonworks-elastic-searchfinal

How would one implement Hadoop and Elasticsearch on the same cluster?
Would there be any trouble between Hadoop ZooKeeper and Elasticsearch's own cluster management?

Thanks to all for paying attention!

Charles

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5c186974-78b9-4d34-9271-ca32b653c5bb%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5c186974-78b9-4d34-9271-ca32b653c5bb%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/54B651D3.1070106%40gmail.com.
For more options, visit https://groups.google.com/d/optout.