# Slow index speed for larger amounts of data

**URL:** https://discuss.elastic.co/t/slow-index-speed-for-larger-amounts-of-data/17958
**Category:** Elasticsearch
**Created:** [June 6, 2014, 5:00pm UTC](https://discuss.elastic.co/t/slow-index-speed-for-larger-amounts-of-data/17958 "2014-06-06T17:00:27Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![edegraff](https://avatars.discourse-cdn.com/v4/letter/e/45deac/32.png) [@edegraff](https://discuss.elastic.co/u/edegraff)
#### Post date: [June 6, 2014, 5:00pm UTC](https://discuss.elastic.co/t/slow-index-speed-for-larger-amounts-of-data/17958/1 "2014-06-06T17:00:27Z")

</div>

The amount of log data we have has recently jumped up to ~700000 rows per minute, and our elasticsearch cluster is no longer able to process the index requests quickly enough(takes about 10 minutes for 1 minute of data).

Our current setup had 5 shards on 5 nodes with 8 cores and 56GB of RAM each. I've tried increasing the shard size to 10 and adding a couple more nodes with no noticeable improvement.

Here are the settings I've tried tweaking so far:

http.max\_content\_length: 1000mb  
indices.memory.index\_buffer\_size: 30%  
indices.memory.min\_shard\_index\_buffer\_size: 12mb  
indices.memory.min\_index\_buffer\_size: 96mb  
index.refresh\_interval: 15s

Any important settings I'm missing? Do we need more nodes/shards?

Thanks!  
Evan

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:24am UTC](https://discuss.elastic.co/t/slow-index-speed-for-larger-amounts-of-data/17958/2 "2017-07-06T01:24:08Z")

</div>