# Optimizing configuration for ingestion

**URL:** <https://discuss.elastic.co/t/optimizing-configuration-for-ingestion/108576>\
**Category:** Elasticsearch\
**Created:** [November 21, 2017, 2:52pm UTC](https://discuss.elastic.co/t/optimizing-configuration-for-ingestion/108576 "2017-11-21T14:52:32Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![eyang](https://avatars.discourse-cdn.com/v4/letter/e/b9e5f3/32.png) [@eyang](https://discuss.elastic.co/u/eyang)\
**Post date:** [November 21, 2017, 2:52pm UTC](https://discuss.elastic.co/t/optimizing-configuration-for-ingestion/108576/1 "2017-11-21T14:52:32Z")

</div>

Hi all,

I've been experimenting with my 3 nodes cluster with a focus on pushing the ingestion performance. The data set I use has 3 million lines, ~840MB, going in through logstash. Although I get decent performance ingesting into an empty index, getting ~6k lines/second, the ingestion slows down as the number of documents in the index grow. After a while I see in the logstash log file entries indicating the elasticsearch ingestion endpoint is not responding. Looking on the motoring tab and running REST query calls with Postman, I see the number of segments fluctuates constantly, seemingly indicating frequent segments merging, which I thought can get expensive as the size of the segments grow and cause elasticsearch to throttle down ingestion.

I do have my 3 hosts on VMs sharing a spinning disk managed by VMware ESXi, but before I try to switch to SSD for datastore, does anybody has any suggestion on how I can debug on the elasticsearch side and narrow down or confirm the cause of my slowing ingestion performance?

Thanks,  
Eric

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 19, 2017, 2:52pm UTC](https://discuss.elastic.co/t/optimizing-configuration-for-ingestion/108576/2 "2017-12-19T14:52:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
