# Slow Elasticsearch Ingestion

**URL:** <https://discuss.elastic.co/t/slow-elasticsearch-ingestion/243006>\
**Category:** Elasticsearch\
**Created:** [July 29, 2020, 6:36am UTC](https://discuss.elastic.co/t/slow-elasticsearch-ingestion/243006 "2020-07-29T06:36:23Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![evacch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evacch/32/49448_2.png) [@evacch](https://discuss.elastic.co/u/evacch)\
**Post date:** [July 29, 2020, 6:36am UTC](https://discuss.elastic.co/t/slow-elasticsearch-ingestion/243006/1 "2020-07-29T06:36:23Z")

</div>

I am new to ELK stack and I am currently testing on the performance of the Elasticsearch with 3 different configurations as follows:

1. One source file: 1GB
2. One source file: 3GB
3. 3 source files: 1GB+1GB+1GB

It took me 3 to 4 hours to have the 3GB file ingested into Elasticsearch. It took me more than a day to finish 3 different 1GB files to be ingested. The speed was getting slower as the size of the index increases.

I have tried to increase the JVM heap size and it's getting slightly better, shortening the time by about one hour. All other settings are configured in the default settings.

I would like to know the additional ways on how to improve the ingestion speed on this? Thanks.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 29, 2020, 6:56am UTC](https://discuss.elastic.co/t/slow-elasticsearch-ingestion/243006/2 "2020-07-29T06:56:05Z")

</div>

What is the specification of your cluster in terms of nodes and hardware used How are you infesting the data?

---

<div class="post-metadata">

**Author:** ![evacch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evacch/32/49448_2.png) [@evacch](https://discuss.elastic.co/u/evacch)\
**Post date:** [July 29, 2020, 7:53am UTC](https://discuss.elastic.co/t/slow-elasticsearch-ingestion/243006/3 "2020-07-29T07:53:44Z")

</div>

This is my current configuration of Elasticsearch. All are in default settings.  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/5/953448feffc6506aa372f2603e057c89f4c56d07.png)

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 29, 2020, 8:00am UTC](https://discuss.elastic.co/t/slow-elasticsearch-ingestion/243006/4 "2020-07-29T08:00:14Z")

</div>

What is the specification of the host? What type of disk/storage do you have?

---

<div class="post-metadata">

**Author:** ![evacch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evacch/32/49448_2.png) [@evacch](https://discuss.elastic.co/u/evacch)\
**Post date:** [July 29, 2020, 8:15am UTC](https://discuss.elastic.co/t/slow-elasticsearch-ingestion/243006/5 "2020-07-29T08:15:07Z")

</div>

I am using VM here in an ESXI server so the hardware specification is as follows:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/6/c6d17ed417ebc9f631302173d4aa01cec754167e.png)

As I need this ELK for only one application so I didn't allocate high hard spec for this VM.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 29, 2020, 8:20am UTC](https://discuss.elastic.co/t/slow-elasticsearch-ingestion/243006/6 "2020-07-29T08:20:50Z")

</div>

Indexing is I/O intensive so having fast storage is important for optimal indexing speed. I would recommend running ‘iostat -x’ on the node while indexing.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 26, 2020, 8:20am UTC](https://discuss.elastic.co/t/slow-elasticsearch-ingestion/243006/7 "2020-08-26T08:20:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
