Combine elasticsearch/logstash/kibana with hadoop


(kay kay) #1

Hi

For performance improvement I'm trying to combine
elasticsearch/logstash/kibana with hadoop (cdh4). Unfortunately I'm
familiar only with HDFS where I store logs. In my opinion the combination
of elasticsearch and hadoop should use hdfs as storage and transparent
hadoop map/reduce functionality for search.

I ran through elasticsearch-hadoop documentation and unfortunately I didn't
understand how this combination could help me for kibana log analyze.
Documentation says "Elasticsearch real-time search and analytics natively
integrated with Hadoop.". But what should I configure? Hadoop with
Elasticsearch or Elasticsearch with Hadoop? As for first one, I found only
java code parts, nothing about the Hadoop configuration, so it seems that I
should be familiar with java programming. As for the last one I found only
"Hadoop HDFS Snapshot/Restore plugin", but I guess it was developed for
indexes backup/restore, am I right?

Anyway, are my expectations right? Or elasticsearch-hadoop was developed
for developers only and it is not suitable for
"elasticsearch/logstash/kibana + hadoop"

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/979a5788-1f17-4351-8c36-e205bc67dca0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #2