# Elasticsearch is taking too long between a BulkImport and another **URL:** \ **Category:** Elasticsearch\ **Created:** [November 29, 2018, 9:31am UTC](https://discuss.elastic.co/t/elasticsearch-is-taking-too-long-between-a-bulkimport-and-another/158720 "2018-11-29T09:31:17Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![andreatera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andreatera/32/115921_2.png) [@andreatera](https://discuss.elastic.co/u/andreatera)\ **Post date:** [November 29, 2018, 9:31am UTC](https://discuss.elastic.co/t/elasticsearch-is-taking-too-long-between-a-bulkimport-and-another/158720/1 "2018-11-29T09:31:18Z")
Hi all, we have a microservice application that is getting data from Kafka and importing them into Elasticsearch using BulkImport operation. - The microservice application is running in docker using docker-compose and scaling for parallel and multi-thread. - Elasticsearch (2.4.1) is also running into docker using docker-compose with the following configuration (1master with 4GB javaHeapSize - 1client with 4GB javaHeapSize - 5data with 8GB javaHeapSize - 24shards - 1index~7.89GB). - The VM have 256GB of RAM, 24 CPU (24core), 500GB disk space ext4 We noted that the application is taking 20s between some BulkImport and continuing with others, at the end to import a fullIndex of 7.49GB (6,2Milions hits) is taking 4h40m.. Not what we expected. We already tried to: 1. Disable refresh and replicas for initial loads 2. Setting ulimits higher 3. Setting scale configuration of threadpools No luck. Can we have some suggestion in order to increase indexing speed? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 29, 2018, 9:39am UTC](https://discuss.elastic.co/t/elasticsearch-is-taking-too-long-between-a-bulkimport-and-another/158720/2 "2018-11-29T09:39:57Z")
I would recommend looking at [this guide](https://www.elastic.co/guide/en/elasticsearch/reference/6.5/tune-for-indexing-speed.html). You may also want to [optimise your mappings](https://www.elastic.co/guide/en/elasticsearch/reference/6.5/tune-for-disk-usage.html) to reduce work required at indexing time. Having a single master-eligible node makes it a single point of failure and is not recommended. Also make sure you are sending data directly to the data nodes so the client node does not become a bottleneck. ---
**Author:** ![andreatera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andreatera/32/115921_2.png) [@andreatera](https://discuss.elastic.co/u/andreatera)\ **Post date:** [November 29, 2018, 2:33pm UTC](https://discuss.elastic.co/t/elasticsearch-is-taking-too-long-between-a-bulkimport-and-another/158720/3 "2018-11-29T14:33:41Z")
Thanks for the fast reply @Christian_Dahlqvist, this env is just a prototype in order to get some metrics useful for prod, were we have 3 clients, 3 master, 12 data nodes.. Anyway good point! Do you know if client node is limiting number of requests/rate? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 29, 2018, 3:00pm UTC](https://discuss.elastic.co/t/elasticsearch-is-taking-too-long-between-a-bulkimport-and-another/158720/4 "2018-11-29T15:00:28Z")
Have a look at the Elasticsearch nodes and see if you can identify what is limiting throughput. Elasticsearch is often very disk I/O intensive, so slow storage is a common bottleneck, but it could also be CPU or GC. ---
**Author:** ![andreatera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andreatera/32/115921_2.png) [@andreatera](https://discuss.elastic.co/u/andreatera)\ **Post date:** [November 29, 2018, 3:27pm UTC](https://discuss.elastic.co/t/elasticsearch-is-taking-too-long-between-a-bulkimport-and-another/158720/5 "2018-11-29T15:27:53Z")
Is not for sure CPU and GC, I already monitored it and it's ok.. Regarding disk I/O we are using volumes with default docker driver. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 29, 2018, 3:32pm UTC](https://discuss.elastic.co/t/elasticsearch-is-taking-too-long-between-a-bulkimport-and-another/158720/6 "2018-11-29T15:32:21Z")
What kind of storage do you have? ---
**Author:** ![andreatera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andreatera/32/115921_2.png) [@andreatera](https://discuss.elastic.co/u/andreatera)\ **Post date:** [November 29, 2018, 4:28pm UTC](https://discuss.elastic.co/t/elasticsearch-is-taking-too-long-between-a-bulkimport-and-another/158720/7 "2018-11-29T16:28:37Z")
VM is running in VMWare vCloud Director with the so called "Fast Storage with Snapshot Site B2".. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 29, 2018, 4:37pm UTC](https://discuss.elastic.co/t/elasticsearch-is-taking-too-long-between-a-bulkimport-and-another/158720/8 "2018-11-29T16:37:06Z")
I have no idea what that means or corresponds to. ---
**Author:** ![andreatera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andreatera/32/115921_2.png) [@andreatera](https://discuss.elastic.co/u/andreatera)\ **Post date:** [November 29, 2018, 4:41pm UTC](https://discuss.elastic.co/t/elasticsearch-is-taking-too-long-between-a-bulkimport-and-another/158720/9 "2018-11-29T16:41:01Z")
Sorry me neither.. I have no more detailed information about this Cloud Storage. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 29, 2018, 4:53pm UTC](https://discuss.elastic.co/t/elasticsearch-is-taking-too-long-between-a-bulkimport-and-another/158720/10 "2018-11-29T16:53:58Z")
Look at iostat on the VM while it is under load, and see if that gives any indication. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 27, 2018, 4:54pm UTC](https://discuss.elastic.co/t/elasticsearch-is-taking-too-long-between-a-bulkimport-and-another/158720/11 "2018-12-27T16:54:05Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.