# How to do bulk insert from Hive to Elasticsearch?

**URL:** <https://discuss.elastic.co/t/how-to-do-bulk-insert-from-hive-to-elasticsearch/54936>\
**Category:** Elasticsearch\
**Created:** [July 7, 2016, 10:28am UTC](https://discuss.elastic.co/t/how-to-do-bulk-insert-from-hive-to-elasticsearch/54936 "2016-07-07T10:28:06Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ravi\_yadav](https://avatars.discourse-cdn.com/v4/letter/r/e9bcb4/32.png) [@ravi\_yadav](https://discuss.elastic.co/u/ravi_yadav)\
**Post date:** [July 7, 2016, 10:28am UTC](https://discuss.elastic.co/t/how-to-do-bulk-insert-from-hive-to-elasticsearch/54936/1 "2016-07-07T10:28:06Z")

</div>

I know that there is a bulk API in ES which can load data from a file, stating that file should have metadata/action before actual data for all the records.

However my use case is to load data from Hive to ES which i'm doing by creating an External Hive table with ES serde. How can I leverage ES bulk load API to load data faster from Hive to ES?

I'm using ES2.1

Thanks!  
Ravi

---

<div class="post-metadata">

**Author:** ![cbuescher](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cbuescher/32/60402_2.png) [@cbuescher](https://discuss.elastic.co/u/cbuescher)\
**Post date:** [July 7, 2016, 10:49am UTC](https://discuss.elastic.co/t/how-to-do-bulk-insert-from-hive-to-elasticsearch/54936/2 "2016-07-07T10:49:46Z")

</div>

Hi,

are you aware of the [Hive Integration](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/hive.html) in ES Hadoop? I'm not too familiar with it so I don't know if this might solve your problem. If not, it would be interesting to know why.

---

<div class="post-metadata">

**Author:** ![ravi\_yadav](https://avatars.discourse-cdn.com/v4/letter/r/e9bcb4/32.png) [@ravi\_yadav](https://discuss.elastic.co/u/ravi_yadav)\
**Post date:** [July 7, 2016, 1:22pm UTC](https://discuss.elastic.co/t/how-to-do-bulk-insert-from-hive-to-elasticsearch/54936/3 "2016-07-07T13:22:29Z")

</div>

Yes, I'm aware of Hive Integration with ES. We are using it to push data successfully to ES from Hive. But the process takes hours. I wanted to know whether bulk load API can improve performance of data load to ES from Hive. If yes, how to use it with Hive? I couldn't find any documentation on that.

---

<div class="post-metadata">

**Author:** ![cbuescher](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cbuescher/32/60402_2.png) [@cbuescher](https://discuss.elastic.co/u/cbuescher)\
**Post date:** [July 7, 2016, 8:36pm UTC](https://discuss.elastic.co/t/how-to-do-bulk-insert-from-hive-to-elasticsearch/54936/4 "2016-07-07T20:36:45Z")

</div>

btw. there is a dedicated sub-forum for Hadoop-related questions like this at [https://discuss.elastic.co/c/elasticsearch-and-hadoop](https://discuss.elastic.co/c/elasticsearch-and-hadoop), maybe folks there have more of an opinion about this question than here in the Elasticsearch forum.

---

<div class="post-metadata">

**Author:** ![ravi\_yadav](https://avatars.discourse-cdn.com/v4/letter/r/e9bcb4/32.png) [@ravi\_yadav](https://discuss.elastic.co/u/ravi_yadav)\
**Post date:** [July 8, 2016, 2:18am UTC](https://discuss.elastic.co/t/how-to-do-bulk-insert-from-hive-to-elasticsearch/54936/5 "2016-07-08T02:18:00Z")

</div>

Reposted there.. Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:37pm UTC](https://discuss.elastic.co/t/how-to-do-bulk-insert-from-hive-to-elasticsearch/54936/6 "2017-07-05T22:37:13Z")

</div>