# The indexing speed become more and more slowly

**URL:** https://discuss.elastic.co/t/the-indexing-speed-become-more-and-more-slowly/10987
**Category:** Elasticsearch
**Created:** [March 4, 2013, 8:48am UTC](https://discuss.elastic.co/t/the-indexing-speed-become-more-and-more-slowly/10987 "2013-03-04T08:48:05Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Hoony](https://avatars.discourse-cdn.com/v4/letter/h/7ab992/32.png) [@Hoony](https://discuss.elastic.co/u/Hoony)
#### Post date: [March 4, 2013, 8:48am UTC](https://discuss.elastic.co/t/the-indexing-speed-become-more-and-more-slowly/10987/1 "2013-03-04T08:48:05Z")

</div>

Hi all,

```
  My elasticsearch indexing become more and more slowly by the size of 

```

index increase.  
e.x.  
The 1 GB file indexing need 10 min at the beginning, and it become  
more and more slowly.  
like here:  
1time 1GB file ------------- 10min indexing  
2time 1GB file ------------- 12min indexing  
3time 1GB file ------------- 14min indexing

```
  I found the indexing rate became 1 MB/s near the end.

  Is it normal? 

```

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)
#### Post date: [March 4, 2013, 9:11am UTC](https://discuss.elastic.co/t/the-indexing-speed-become-more-and-more-slowly/10987/2 "2013-03-04T09:11:33Z")

</div>

it's not 🙂 can we get more info about your cluster / setup. I am wondering  
about things like:

- indexing rate
- are you doing bulk
- memory settings
- hardware configuration
- any special settings like merge throtteling or custom merge policies
- are you searching on the same cluster at the same time?

simon

On Monday, March 4, 2013 9:48:05 AM UTC+1, Hoony wrote:

> Hi all,
> 
> ```
> My elasticsearch indexing become more and more slowly by the size of 
> 
> ```
> 
> index increase.  
> e.x.  
> The 1 GB file indexing need 10 min at the beginning, and it become  
> more and more slowly.  
> like here:  
> 1time 1GB file ------------- 10min indexing  
> 2time 1GB file ------------- 12min indexing  
> 3time 1GB file ------------- 14min indexing
> 
> ```
> I found the indexing rate became 1 MB/s near the end.
> 
> Is it normal? 
> 
> ```

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![Hoony](https://avatars.discourse-cdn.com/v4/letter/h/7ab992/32.png) [@Hoony](https://discuss.elastic.co/u/Hoony)
#### Post date: [March 5, 2013, 1:53am UTC](https://discuss.elastic.co/t/the-indexing-speed-become-more-and-more-slowly/10987/3 "2013-03-05T01:53:49Z")

</div>

My cluster environment here:

- indexing rate - I am not sure what you need, because the rate is not  
established and the speed becomes more and more slow , as I said before.
- are you doing bulk - I used bulk and the bulk size is 2000.
- memory settings - ram is 16GB and configured 8GB for ES
- hardware configuration - CPU 4core 3.3 ghz
- any special settings like merge throtteling or custom merge policies
- I used the default merge setting.
- are you searching on the same cluster at the same time? - I just  
indexing without searching on my cluster at the same time.

在 2013年3月4日星期一UTC+9下午6时11分33秒，simonw写道：

> it's not 🙂 can we get more info about your cluster / setup. I am  
> wondering about things like:
> 
> - indexing rate
> - are you doing bulk
> - memory settings
> - hardware configuration
> - any special settings like merge throtteling or custom merge policies
> - are you searching on the same cluster at the same time?
> 
> simon
> 
> On Monday, March 4, 2013 9:48:05 AM UTC+1, Hoony wrote:
> 
> > Hi all,
> > 
> > ```
> > My elasticsearch indexing become more and more slowly by the size 
> > 
> > ```
> > 
> > of index increase.  
> > e.x.  
> > The 1 GB file indexing need 10 min at the beginning, and it become  
> > more and more slowly.  
> > like here:  
> > 1time 1GB file ------------- 10min indexing  
> > 2time 1GB file ------------- 12min indexing  
> > 3time 1GB file ------------- 14min indexing
> > 
> > ```
> > I found the indexing rate became 1 MB/s near the end.
> > 
> > Is it normal? 
> > 
> > ```

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)
#### Post date: [March 5, 2013, 2:56am UTC](https://discuss.elastic.co/t/the-indexing-speed-become-more-and-more-slowly/10987/4 "2013-03-05T02:56:03Z")

</div>

Hi,

I think Hoony may be seeing the effect of Lucene segment merging, no?

We just did some ES indexing benchmarking and observed the same thing.

## Otis

ELASTICSEARCH Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html)

On Monday, March 4, 2013 4:11:33 AM UTC-5, simonw wrote:

> it's not 🙂 can we get more info about your cluster / setup. I am  
> wondering about things like:
> 
> - indexing rate
> - are you doing bulk
> - memory settings
> - hardware configuration
> - any special settings like merge throtteling or custom merge policies
> - are you searching on the same cluster at the same time?
> 
> simon
> 
> On Monday, March 4, 2013 9:48:05 AM UTC+1, Hoony wrote:
> 
> > Hi all,
> > 
> > ```
> > My elasticsearch indexing become more and more slowly by the size 
> > 
> > ```
> > 
> > of index increase.  
> > e.x.  
> > The 1 GB file indexing need 10 min at the beginning, and it become  
> > more and more slowly.  
> > like here:  
> > 1time 1GB file ------------- 10min indexing  
> > 2time 1GB file ------------- 12min indexing  
> > 3time 1GB file ------------- 14min indexing
> > 
> > ```
> > I found the indexing rate became 1 MB/s near the end.
> > 
> > Is it normal? 
> > 
> > ```

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:48am UTC](https://discuss.elastic.co/t/the-indexing-speed-become-more-and-more-slowly/10987/5 "2017-07-06T02:48:23Z")

</div>


