Pulling data from HDFS to elasticsearch

(bowow) #1

Hi everyone,
How do i read data from HDFS..then send them to elasticsearch and display on kibana dashboard?

I know there is a connector in ES-hadoop which works bidirectionally to to send data out/recieve....but my question is in the first stage Pulling data from HDFS

What are the most feasible options? Please provide me with couple of options..

(Costin Leau) #2

If you are using Hadoop, then you can use the various libraries out there such as Map/Reduce, Hive, Pig, Spark, Cascading.
If you are not, then using any client that connects to HDFS will do (Logstash for example).

(system) #3