Hive external table automatically send data to elasticsearch

(Vag) #1


I'm new to elasticsearch and es-hadoop and I have this problem: I managed to connect hive with elasticsearch and with the command insert overwrite table i can pass data form an external hive table to external hive-elasticsearch table but i want to know if there is a way to automatically pass data to elasticsearch when there are new data into hdfs files. The hive external table(not elasticsearch) takes data from files into hdfs with the command location '/path/to/file'; automatically. How can i do it with elasticsearch external table. This command doesn't seem to work with elasticsearch.

Thank you

(Costin Leau) #2

That's because an external table can be backed by sources other than HDFS. In fact, with the connector the table is backed by an Elasticsearch index - hence if you want data to be available in Elasticsearch, you need to index it.

(system) #3