Elasticserch slowly inserts a row

Elastic Stack Elasticsearch
1

We have got a problem with elasticsearch. When we used version 1.7.0 of elasticsearch the speed of insertion a row was approximately 0.01 sec but when we updated version on 2.3.1 the speed of insertion a row slowed down approximately in 10 times, i.e. it became about 0.1 second. Our OS: Ubuntu 14.04 LTS

java version "1.7.0_101" OpenJDK Runtime Environment (IcedTea 2.6.6) (7u101-2.6.6-0ubuntu0.14.04.1) OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

also we have tried with:

java version "1.8.0_91" Java(TM) SE Runtime Environment (build 1.8.0_91-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

but we have got same results with speed.

mapping: {"properties":{"id_item":{"type":"long"},"id_position_to_item":{"type":"integer"},"cost":{"type":"long"},"cost_per":{"type":"long"},"id_country_to_item":{"type":"integer"},"id_oblast_to_item":{"type":"integer"},"id_raion_to_item":{"type":"integer"},"id_city_to_item":{"type":"integer"},"id_metro":{"type":"integer"},"id_borough":{"type":"integer"},"id_borough_sup":{"type":"integer"},"id_way":{"type":"integer"},"street_id":{"type":"integer"},"time_create":{"type":"long"},"time_add":{"type":"long"},"time_delete":{"type":"long"},"square_item":{"type":"float"},"count_room":{"type":"integer"},"mark_add_day":{"type":"long"},"mark_del_day":{"type":"long"},"mark_pub_day":{"type":"long"}}}

row insert: {"id_position_to_item":"499","id_item":"3","cost":"3105000","cost_per":"3105000","id_country_to_item":"1","id_oblast_to_item":"26","id_raion_to_item":"0","id_city_to_item":"499","id_metro":"0","id_borough":"0","id_borough_sup":"0","id_way":"17","street_id":"220920","square_item":"35.1","count_room":"2","id_for_es":"3","mark_add_day":1459371600,"mark_del_day":1459630800,"mark_pub_day":[1459371600,1459458000,1459544400,1459630800]}

2

It's not a row, it's a document.

However it is probably due to the fact that we fsync the translog on every write operation in 2.X.
Are you using bulk?

3

No, I don't use bulk.
Does the problem connect with "fsync and commit after every request"? Because commit had been every 5 seconds in version 1.7.
So according to you advice should I set index.translog.durability = async?

4

Setting durability = async puts you in a situation that you could lose data, so a better option would be to group indexing requests together in larger bulk requests. For instance if you can put 1000 indexing requests in a single bulk, elasticsearch will perform ${num_shards} fsync calls instead of 1000.

1 Like