I am using HDFS to store data (files1.txt), and load them into a Hive table (user_source). Then I create a external table as below:
create external table user_tr( id int,name string) row format delimited fields terminated by ',' lines terminated by '\n' stored by 'org.elasticsearch.hadoop.hive.EsStorageHandler' location '/user/hive/warehouse/user_source' tblproperties ( 'es.resource'='/blogs/user/', 'es.index.auto.create'='true', 'es.nodes'='172.x.y.z', 'es.port'='7200', 'es.mapping.names'='id:id,name:name');
and the table is created successfully.
However, in the /_plugin/head/, I cannot see the type created automaticlly under the 'blogs' index.
Of course, if i create an external table stored by 'org.elasticsearch.hadoop.hive.EsStorageHandler', and do command :
"insert overwrite table user_try
select * from user_source;". There will be a type created and I can see all data is loaded in the ES.
Is that because the location command is not applicable? If so, how can I automatically load data into ES without create a external table and insert data into the table from a Hive table?
Looking forward for reply.
PS: the whole problem is about: I used sqoop to load data from mysql to HDFS, it worked. Then I wanted to create a external table mapped to the HDFS location.
So, I can avoid creating a local Hive table, and use 'insert' to load data from the Hive table to the external one.
But, every time I sqoop more data, the es index size increases.