Real example with elasticsearch and hadoop?


(Thomas Decaux) #1

Hello,

I am looking for real examples (full code) to play with elasticsearch and hadoop.

Especially about data mapping (on Pig and Spark).

Thanks you,


(Jimmy Kuang) #2

I found one example on Spark and Hadoop here.

https://www.elastic.co/guide/en/elasticsearch/hadoop/current/spark.html


(Thomas Decaux) #3

Not really "real" ^^

Actually I found no one who is using it really , search also on github, no big PIG file with elasticsearch (I could miss something for sure).


(Jimmy Kuang) #4

^^ Depends what you mean by real. Are you looking for a story line and examples of how that story line is implemented with ES and hadoop?

I just installed the ES-hadoop connector and Spark. Then copy and paste the example in the doc and it all works. So from that, real example meant it actually works like the doc describe to me.


(Thomas Decaux) #5

Sure they work ! 2 lines of code, I hope so ^^

I was searching for real use case, such as "use elasticsearch with Spark machine learning" or "export elasticsearch data with PIG and save it as parquet file".

As I am more "elasticsearch" than Hadoop, I was expecting something like that. Now I have worked with a Hadoop guy, so it's fine.

Thanks you,


(system) #6