Kibana that supports elasticsearch-hadoop 2.4.0

Elastic Stack Elasticsearch
1

I am a newbee to Elastic. I am using elasticsearch-hadoop-2.4.0 with pyspark. And I can successfully write to and read from elasticsearch from within pyspark. Now I want to visualize the data in Kibana. When I start Kibana 4.6.1, I get the error message that says "This version of Kibana requires Elasticsearch ^2.4.0 on all nodes. I found the following incompatible nodes in your cluster: Elasticsearch v1.2.1".

How can I fix this issue? Is it possible to upgrade the elasticsearch version within the elasticsearch-hadoop-2.4.0? Or what is the latest version of Kibana that is compatible with elasticsearch 1.2.1?

Thanks in advance!

2

Hi,

ES-Hadoop is a library installed on Hadoop only, so your version issues have no direct relation to this error message. Can you go around all nodes in your ES cluster and check the version? The easiest way to do that is to do this GET:

curl -XGET [host]:[port]

It should come back like this:

{
  "status" : 200,
  "name" : "Big Man",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "1.7.2",
    "build_hash" : "e43676b1385b8125d647f593f7202acbd816e8ec",
    "build_timestamp" : "2015-09-14T09:49:53Z",
    "build_snapshot" : false,
    "lucene_version" : "4.10.4"
  },
  "tagline" : "You Know, for Search"
}
3

Hi Loek,

the message I get when I open the :9200 port is shown below.

FYI, I have not explicitly installed elasticsearch on my node. I just downloaded the elasticsearch-hadoop-2.4.0.zip, unzipped it and added the the ".../elasticsearch-hadoop-2.4.0/dist/elasticsearch-hadoop-2.4.0.jar" to my spark class path.

{
"status" : 200,
"name" : "Cerberus",
"version" : {
"number" : "1.2.1",
"build_hash" : "6c95b759f9e7ef0f8e17f77d850da43ce8a4b364",
"build_timestamp" : "2014-06-03T15:02:52Z",
"build_snapshot" : false,
"lucene_version" : "4.8"
},
"tagline" : "You Know, for Search"
}

4

This means that there is a version of ES running on the machine, and it's an old version that you should not use right now for something new. It might be part of another piece of software that depends on ES or something like that. You could try

curl -XGET <host>:<port>/_cat/indices

...to see what is actually in there.

You might want to install ES 2.4.x from our website and have it run on another port then 9200 if you're not sure about where this ES came from.

5

Thanks Loek!

I have checked the indices, and they are only the ones I created when I wrote to elasticsearch from within pyspark.

My guess is that (I could be wrong) when the elasticsearch-hadoop library is called from pyspark, it starts an instance of elasticsearch. In this case, it seems that it is staritng elasticsearch 1.2.1. Is that possible?

6

I'm not too familiar with that lib, but I guess it's possible. It is a truly outdated version and you should deploy ES 2.4.x instead, probably just disable this ES first. Let me know how it works out!

7

Ok. I will try to deploy ES 2.4.x and see what comes out. Thanks!