# Low performance on bulk insert with custom mapping

**URL:** <https://discuss.elastic.co/t/low-performance-on-bulk-insert-with-custom-mapping/154899>\
**Category:** Elasticsearch\
**Created:** [October 31, 2018, 5:36pm UTC](https://discuss.elastic.co/t/low-performance-on-bulk-insert-with-custom-mapping/154899 "2018-10-31T17:36:28Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![0011001011](https://avatars.discourse-cdn.com/v4/letter/0/bb73d2/32.png) [@0011001011](https://discuss.elastic.co/u/0011001011)\
**Post date:** [October 31, 2018, 5:36pm UTC](https://discuss.elastic.co/t/low-performance-on-bulk-insert-with-custom-mapping/154899/1 "2018-10-31T17:36:28Z")

</div>

Hi all,

I encountered a strange problem with ES 6.2.4. I try to bulk insert data into my ES nodes, which run fine, but which is also slow ( **~10s** per 1000 insert).

I run it on 4go of RAM, the CPU is used at ~24%, with 3 shards, and refresh deactivated.

If I do not specify a mapping and let ES dynamically finds it, I get only **1s** to index the bulk of 1000 docs. Alright, my mapping is complex and lots of preprocesing.

Now what is strange is that if i retrieve the mapping dynamically provided by ES (through `GET /my-index/`) and then use this **exact same mapping** on a new instance, I also have the bad perf ( **~10s per bulk** ) while it should be the same.

Any idea on that ? I feel like I am missing something here and it's kind of frustrating 😕

Regards

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 31, 2018, 5:41pm UTC](https://discuss.elastic.co/t/low-performance-on-bulk-insert-with-custom-mapping/154899/2 "2018-10-31T17:41:53Z")

</div>

What type of hardware are you running on? What type of storage? What is the size of your documents?

---

<div class="post-metadata">

**Author:** ![0011001011](https://avatars.discourse-cdn.com/v4/letter/0/bb73d2/32.png) [@0011001011](https://discuss.elastic.co/u/0011001011)\
**Post date:** [October 31, 2018, 5:50pm UTC](https://discuss.elastic.co/t/low-performance-on-bulk-insert-with-custom-mapping/154899/3 "2018-10-31T17:50:55Z")

</div>

Running on classical HDD, Xeon E5-2680 (2.40Ghz)

One document is 50 fields long, each field is "short" (less than 50 characters)

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 31, 2018, 6:03pm UTC](https://discuss.elastic.co/t/low-performance-on-bulk-insert-with-custom-mapping/154899/4 "2018-10-31T18:03:01Z")

</div>

That should work the same way. Are you 100% sure that you don't have something else like an index template that changes the mapping ?

---

<div class="post-metadata">

**Author:** ![0011001011](https://avatars.discourse-cdn.com/v4/letter/0/bb73d2/32.png) [@0011001011](https://discuss.elastic.co/u/0011001011)\
**Post date:** [October 31, 2018, 6:07pm UTC](https://discuss.elastic.co/t/low-performance-on-bulk-insert-with-custom-mapping/154899/5 "2018-10-31T18:07:25Z")

</div>

Yep I am positive on that.

---

<div class="post-metadata">

**Author:** ![0011001011](https://avatars.discourse-cdn.com/v4/letter/0/bb73d2/32.png) [@0011001011](https://discuss.elastic.co/u/0011001011)\
**Post date:** [October 31, 2018, 6:08pm UTC](https://discuss.elastic.co/t/low-performance-on-bulk-insert-with-custom-mapping/154899/6 "2018-10-31T18:08:23Z")

</div>

Btw for what it is worth, I tried to create an empty mapping  
{ myindexname : {} }  
And then I lost the perf...

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [October 31, 2018, 6:23pm UTC](https://discuss.elastic.co/t/low-performance-on-bulk-insert-with-custom-mapping/154899/7 "2018-10-31T18:23:21Z")

</div>

Can you try with 6.4.2?

---

<div class="post-metadata">

**Author:** ![0011001011](https://avatars.discourse-cdn.com/v4/letter/0/bb73d2/32.png) [@0011001011](https://discuss.elastic.co/u/0011001011)\
**Post date:** [October 31, 2018, 6:56pm UTC](https://discuss.elastic.co/t/low-performance-on-bulk-insert-with-custom-mapping/154899/8 "2018-10-31T18:56:50Z")

</div>

I am currently out of work, and i dont know if it is available with the proxy/artifactory. I will check that asap. Is it a know bug ?

---

<div class="post-metadata">

**Author:** ![0011001011](https://avatars.discourse-cdn.com/v4/letter/0/bb73d2/32.png) [@0011001011](https://discuss.elastic.co/u/0011001011)\
**Post date:** [November 5, 2018, 9:28am UTC](https://discuss.elastic.co/t/low-performance-on-bulk-insert-with-custom-mapping/154899/9 "2018-11-05T09:28:52Z")

</div>

Any insight ? I can not test with 6.4.2 unfortunately

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 3, 2018, 9:28am UTC](https://discuss.elastic.co/t/low-performance-on-bulk-insert-with-custom-mapping/154899/10 "2018-12-03T09:28:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
