Elasticsearch Hadoop Connection

Elastic Stack Elasticsearch
1

Hi,

I am trying to add hive table into Elasticsearch. Have added the jar in hive-site.xml file.

CDH 5.11
Hadoop 2.6
Hive 1.1.0
elasticsearch-hadoop-7.3.2.jar
Elasticsearch 7.3.2

I am able to successfully create table using below command
CREATE EXTERNAL TABLE elkdummy (
name STRING,
id INT)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.nodes' = '192.168.4.23', 'es.resource' = 'hive-elastic/log', 'es.query' = '?q=*');

But when I try to insert data using below command

INSERT INTO TABLE elkdummy VALUES ('elastic',1);

I get below error
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.hadoop.hive.EsHiveInputFormat
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hive.com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:136)
... 49 more

I want to understand why the create query is getting executed but the insert query is not?

2

We've seen issues when people add the jar in hive-site.xml - There are weird problems that pop up in Hive when trying to load the library when executing a query. Often times it is resolved by adding the jar to the job classpath via the ADD JAR <...> command before the queries are run.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.