Kibana is very tightly integrated with Elasticsearch, to the point of
requiring specific versions of Elasticsearch for a given version of Kibana.
When you say Hadoop that really means nothing. Most of the Hadoop
EcoSystem is not realtime. There are some exceptions like HBase, but their
characteristics are very different from Elasticsearch. This is why it is
so common to use Elasticsearch with a Hadoop environment, to provide a
realtime aspect to your big data. This is why as you say Elasticsearch is
in the middle. There is not reasonable way Kibana could extract random
bits of data from "Hadoop".
That said there are many people who have built data visualizations using
Hadoop and D3, the js lib that is used in Kibana. If you really want to go
without Elasticsearch (you don't), I would recommend looking for something
like that.
Aaron
On Thursday, March 12, 2015 at 2:41:42 AM UTC-6, KRRK2015 wrote:
Hello, has anyone tried to get Kibana work directly with Hadoop (without
elasticsearch in the middle)? If yes, how? Any references would help.
Thanks.
On the bright side, you can use es-hadoop connector [1] to easily get data
from Hadoop/HDFS to Elasticsearch and back whatever your Hadoop stack
(Map/Reduce, Cascading, Pig, Hive, Spark, Storm).
Kibana is very tightly integrated with Elasticsearch, to the point of
requiring specific versions of Elasticsearch for a given version of Kibana.
When you say Hadoop that really means nothing. Most of the Hadoop
EcoSystem is not realtime. There are some exceptions like HBase, but their
characteristics are very different from Elasticsearch. This is why it is
so common to use Elasticsearch with a Hadoop environment, to provide a
realtime aspect to your big data. This is why as you say Elasticsearch is
in the middle. There is not reasonable way Kibana could extract random
bits of data from "Hadoop".
That said there are many people who have built data visualizations using
Hadoop and D3, the js lib that is used in Kibana. If you really want to go
without Elasticsearch (you don't), I would recommend looking for something
like that.
Aaron
On Thursday, March 12, 2015 at 2:41:42 AM UTC-6, KRRK2015 wrote:
Hello, has anyone tried to get Kibana work directly with Hadoop (without
elasticsearch in the middle)? If yes, how? Any references would help.
Thanks.
Thanks Aaron, Costin for validating what I suspected. Agreed that ES is
realt-time vs. Hadoop. I was looking at Kibana as a standalone data
visualization tool for batch scenarios due to its superior visualization
capabilities. As you said, perhaps D3 is the only option as of now.
Thanks again for your quick response.
On Friday, 13 March 2015 09:29:16 UTC+5:30, Costin Leau wrote:
On the bright side, you can use es-hadoop connector [1] to easily get data
from Hadoop/HDFS to Elasticsearch and back whatever your Hadoop stack
(Map/Reduce, Cascading, Pig, Hive, Spark, Storm).
On Fri, Mar 13, 2015 at 3:15 AM, <aa...@definemg.com <javascript:>> wrote:
Kibana is very tightly integrated with Elasticsearch, to the point of
requiring specific versions of Elasticsearch for a given version of Kibana.
When you say Hadoop that really means nothing. Most of the Hadoop
EcoSystem is not realtime. There are some exceptions like HBase, but their
characteristics are very different from Elasticsearch. This is why it is
so common to use Elasticsearch with a Hadoop environment, to provide a
realtime aspect to your big data. This is why as you say Elasticsearch is
in the middle. There is not reasonable way Kibana could extract random
bits of data from "Hadoop".
That said there are many people who have built data visualizations using
Hadoop and D3, the js lib that is used in Kibana. If you really want to go
without Elasticsearch (you don't), I would recommend looking for something
like that.
Aaron
On Thursday, March 12, 2015 at 2:41:42 AM UTC-6, KRRK2015 wrote:
Hello, has anyone tried to get Kibana work directly with Hadoop (without
elasticsearch in the middle)? If yes, how? Any references would help.
Thanks.
To second what Costin said, it is trivial to load the data into
Elasticsearch, then you have the data ready to use the full power of
Kibana. Elasticsearch is very quick to setup and scales well.
Trying to go the route of rolling your own setup to generate graphs with D3
will certainly be harder, and less rewarding. Just to start with you will
need somewhere to store the large JSON structures you generate from Hadoop
MapReduce. I do know a great JSON store, Elasticsearch. I hope you are
starting to get the idea. In order to produce the web visulizations, you
need to move your data from HDFS to somewhere realtime. That could be
HBase or dozens of other options, but none of them will be as easy or
rewarding as Elasticsearch.
On the bright side, you can use es-hadoop connector [1] to easily get data
from Hadoop/HDFS to Elasticsearch and back whatever your Hadoop stack
(Map/Reduce, Cascading, Pig, Hive, Spark, Storm).
Kibana is very tightly integrated with Elasticsearch, to the point of
requiring specific versions of Elasticsearch for a given version of Kibana.
When you say Hadoop that really means nothing. Most of the Hadoop
EcoSystem is not realtime. There are some exceptions like HBase, but their
characteristics are very different from Elasticsearch. This is why it is
so common to use Elasticsearch with a Hadoop environment, to provide a
realtime aspect to your big data. This is why as you say Elasticsearch is
in the middle. There is not reasonable way Kibana could extract random
bits of data from "Hadoop".
That said there are many people who have built data visualizations using
Hadoop and D3, the js lib that is used in Kibana. If you really want to go
without Elasticsearch (you don't), I would recommend looking for something
like that.
Aaron
On Thursday, March 12, 2015 at 2:41:42 AM UTC-6, KRRK2015 wrote:
Hello, has anyone tried to get Kibana work directly with Hadoop (without
elasticsearch in the middle)? If yes, how? Any references would help.
Thanks.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.