Hive external table automatically send data to elasticsearch

[vage (Vag)]

Hi,

I'm new to elasticsearch and es-hadoop and I have this problem: I managed to connect hive with elasticsearch and with the command insert overwrite table i can pass data form an external hive table to external hive-elasticsearch table but i want to know if there is a way to automatically pass data to elasticsearch when there are new data into hdfs files. The hive external table(not elasticsearch) takes data from files into hdfs with the command location '/path/to/file'; automatically. How can i do it with elasticsearch external table. This command doesn't seem to work with elasticsearch.

Thank you

[costin (Costin Leau)]

@vage

The hive external table(not elasticsearch) takes data from files into hdfs with the command location '/path/to/file'; automatically. How can i do it with elasticsearch external table. This command doesn't seem to work with elasticsearch.

That's because an external table can be backed by sources other than HDFS. In fact, with the connector the table is backed by an Elasticsearch index - hence if you want data to be available in Elasticsearch, you need to index it.