Error: Unsupported/Unknown Elasticsearch version 6.X.X

[spagano (Sergio)]

Hi,

I saw the same problem on too many places.

I have Hortonworks 2.6.4, Hadoop 2.7.3.2.6.4.0-91, Hive 1.2.1000.2.6.4.0-91, and, on the same host, I have installed Elasticsearch 6.4.3.

Then, I do the following

by command line:

1. wget -P /tmp https://artifacts.elastic.co/downloads/elasticsearch-hadoop/elasticsearch-hadoop-6.4.3.zip;
   ok
2. unzip /tmp/elasticsearch-hadoop-6.4.3.zip -d /tmp;
   ok
3. hdfs dfs -copyFromLocal /tmp/elasticsearch-hadoop-6.4.3/dist/elasticsearch-hadoop-6.4.3.jar /tmp
   ok

On beeline o hive from dbeaver I do this:

4. ADD JAR hdfs:///tmp/elasticsearch-hadoop-6.4.3.jar;
   LIST JAR;
   ok

5. CREATE EXTERNAL TABLE artists (
   id BIGINT,
   name STRING,
   links STRUCT<url:STRING, picture:STRING>)
   STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
   TBLPROPERTIES('es.nodes' = 'localhost' , 'es.resource' = 'radio/artists');
   ok

6. select * from artists;
   wrong!

I GET "java.io.IOException: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Unsupported/Unknown Elasticsearch version 6.4.3"

Why?? my ES is 6.4.3 and my ES-Hadoop 6.4.3

I saw the java code from elasticsearch-hadoop project and found this:

For InicializationUtils.java

if (!(esVersion.startsWith("1.") || esVersion.startsWith("2."))) {
throw new EsHadoopIllegalArgumentException("Unsupported/Unknown Elasticsearch version " + esVersion);
}

This is a validation that my ES has to be 1.x or 2.x????

What is wrong here??

Thanks in advance!!!

Sergio.

[spagano (Sergio)]

The problem was solved.

The roadmap was the following:

1. Use hive, no beeline.

2. Just copy elasticsearch-hadoop-6.4.3.jar to all data nodes and masters on /{HDP_HOME}/hadoop/lib/ (hortonworks to me)

3. Create index and type on elasticsearch
   curl -X POST 'localhost:9200/bookindex/books' -H 'Content-Type: application/json' -d'
   {
   "bookId" : "A00-3",
   "author" : "Sankaran",
   "publisher" : "Mcgrahill",
   "name" : "how to get a job"
   }
   '

4. On hive always to do:
   hive> add jar /usr/hdp/2.6.4.0-91/hadoop/lib/commons-httpclient-3.0.1.jar;
   hive> add jar /usr/hdp/2.6.4.0-91/hadoop/lib/elasticsearch-hadoop-6.4.3.jar;
   hive> CREATE EXTERNAL TABLE books ( bookId STRING, author STRING, publisher STRING, name STRING) STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' TBLPROPERTIES('es.nodes.wan.only'= 'TRUE', 'es.nodes.discovery' = 'false', 'es.nodes' = 'localhost', 'es.port' = '9200', 'es.resource' = 'bookindex/books');
   hive> insert into books(bookid,author,publisher,name) values ('2','Pablo Neruda', 'Estrada','Poemas de Pablo Neruda');
   hive> select count(*) from books;
   hive>select * from books;
   OK

5. Check bookindex on elasticsearch!!

Enjoy!

Sergio.