# Elasticsearch bulk insert is too slow on AWS Elasticsearch Service

**URL:** <https://discuss.elastic.co/t/elasticsearch-bulk-insert-is-too-slow-on-aws-elasticsearch-service/113097>\
**Category:** Elasticsearch\
**Created:** [December 24, 2017, 8:20am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-is-too-slow-on-aws-elasticsearch-service/113097 "2017-12-24T08:20:47Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![nmadahi](https://avatars.discourse-cdn.com/v4/letter/n/e99b99/32.png) [@nmadahi](https://discuss.elastic.co/u/nmadahi)\
**Post date:** [December 24, 2017, 8:20am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-is-too-slow-on-aws-elasticsearch-service/113097/1 "2017-12-24T08:20:48Z")

</div>

I setup ES 6.0 Service on AWS using m3.xlarge.elasticsearch, i followed Network Limits ([http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html](http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html)).  
Bulk insert on my local network is too fast (12 min for 4 millions of document) but too slow on AWS (4 hours for same documents).  
I set the replicas to 0 ,refresh\_intervall "-1" and other things that said for performance improvement, but no any luck.

Can anyone help to find the solution?  
There is same issue in [Elasticsearch on aws ec2 index very slow](https://discuss.elastic.co/t/elasticsearch-on-aws-ec2-index-very-slow/38120) but it closed without any succeed message!!!

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 24, 2017, 8:44am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-is-too-slow-on-aws-elasticsearch-service/113097/2 "2017-12-24T08:44:51Z")

</div>

Have you asked AWS? Do you have any monitoring on the nodes?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 24, 2017, 9:57am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-is-too-slow-on-aws-elasticsearch-service/113097/3 "2017-12-24T09:57:28Z")

</div>

How many nodes do you have in your cluster? What is your average document size? What type of data are you indexing?

---

<div class="post-metadata">

**Author:** ![nmadahi](https://avatars.discourse-cdn.com/v4/letter/n/e99b99/32.png) [@nmadahi](https://discuss.elastic.co/u/nmadahi)\
**Post date:** [December 25, 2017, 9:10am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-is-too-slow-on-aws-elasticsearch-service/113097/4 "2017-12-25T09:10:56Z")

</div>

> [@warkolm](#):
>
> ve you asked AW

Not asked from AWS. No, how can monitor nodes on AWS?

---

<div class="post-metadata">

**Author:** ![nmadahi](https://avatars.discourse-cdn.com/v4/letter/n/e99b99/32.png) [@nmadahi](https://discuss.elastic.co/u/nmadahi)\
**Post date:** [December 25, 2017, 9:12am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-is-too-slow-on-aws-elasticsearch-service/113097/5 "2017-12-25T09:12:52Z")

</div>

> [@warkolm](#):
>
> e you asked AWS? Do

I tested with 1 node and 3 nodes. I also tested with different size 1MB, 5MB, 50MB, 90MB, but no success.  
The data are person text information.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 25, 2017, 3:16pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-is-too-slow-on-aws-elasticsearch-service/113097/6 "2017-12-25T15:16:56Z")

</div>

What is the latency to AWS from the machine you are loading from? Are the sizes you specified the average document size you used?

How are you loading data? What is the size of your bulk requests? How many parallell indexing threads/processes are you using?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 22, 2018, 3:16pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insert-is-too-slow-on-aws-elasticsearch-service/113097/7 "2018-01-22T15:16:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
