Create index to HDFS from ElasticSearch

(Roberto Sancho Rojas) #1


I want to index the data from HDFS and show to the kibana.
Actaully elastic and kibana are working in locally directory, but can I search from HDFS JSON files?? How??


(Dylan Johnson) #2

(Roberto Sancho Rojas) #3

Thanks but what i need is store the data and index there like a gateway, but i have readed its deprecated, so i dont need a snapshop or restore.

Wich plugin and configuration i need??

Many thanks

(David Pilato) #4

You need to write a process which will read your data from HDFS, transform to JSON then send to elasticsearch.
Or you can use es-hadoop to do it.

Data needs to be indexed into elasticsearch before you can use Kibana.

I'd suggest looking at this video:

Might help.

(Roberto Sancho Rojas) #5

Actually i need index the data from HDFS and store the data and index also into HDFS, is posible??? i dont want store any file into local filesystem.

I read Gateway but is just to store metadata.


(David Pilato) #6

You could may be mount an HDFS file system on your machine and put your indices in it but it's absolutely not recommended!

(Mark Walkom) #7

Yeah I'd definitely stay away from that.

(system) #8