Low performance on bulk insert with custom mapping

Elastic Stack Elasticsearch
1

Hi all,

I encountered a strange problem with ES 6.2.4. I try to bulk insert data into my ES nodes, which run fine, but which is also slow (~10s per 1000 insert).

I run it on 4go of RAM, the CPU is used at ~24%, with 3 shards, and refresh deactivated.

If I do not specify a mapping and let ES dynamically finds it, I get only 1s to index the bulk of 1000 docs. Alright, my mapping is complex and lots of preprocesing.

Now what is strange is that if i retrieve the mapping dynamically provided by ES (through GET /my-index/) and then use this exact same mapping on a new instance, I also have the bad perf (~10s per bulk) while it should be the same.

Any idea on that ? I feel like I am missing something here and it's kind of frustrating :confused:

Regards

2

What type of hardware are you running on? What type of storage? What is the size of your documents?

3

Running on classical HDD, Xeon E5-2680 (2.40Ghz)

One document is 50 fields long, each field is "short" (less than 50 characters)

4

That should work the same way. Are you 100% sure that you don't have something else like an index template that changes the mapping ?

5

Yep I am positive on that.

6

Btw for what it is worth, I tried to create an empty mapping
{ myindexname : {} }
And then I lost the perf...

7

Can you try with 6.4.2?

8

I am currently out of work, and i dont know if it is available with the proxy/artifactory. I will check that asap. Is it a know bug ?

9

Any insight ? I can not test with 6.4.2 unfortunately