# No efect refresh\_interval

**URL:** https://discuss.elastic.co/t/no-efect-refresh-interval/18694
**Category:** Elasticsearch
**Created:** [July 16, 2014, 10:51am UTC](https://discuss.elastic.co/t/no-efect-refresh-interval/18694 "2014-07-16T10:51:55Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Marek\_Dabrowski](https://avatars.discourse-cdn.com/v4/letter/m/b5ac83/32.png) [@Marek\_Dabrowski](https://discuss.elastic.co/u/Marek_Dabrowski)
#### Post date: [July 16, 2014, 10:51am UTC](https://discuss.elastic.co/t/no-efect-refresh-interval/18694/1 "2014-07-16T10:51:55Z")

</div>

Hello

My configuration is:  
6 nodes Elasticsearch cluster  
OS: Centos 6.5  
JVM: 1.7.0\_25

Cluster is working fine. I can indexing data, query, etc. Now I'm doing  
test on package about ~50mln doc (~13GB). I would like take better  
performance during indexing data. To take this target I has been changed  
parameter refresh\_interval. I did test for 1s, -1 and 600s. Time for  
indexing data is that same. I checked configuration (\_settings) for index  
and value for refresh\_interval is ok (has proper value), eg:

{  
"smt\_20140501\_100000\_20g\_norefresh" : {  
"settings" : {  
"index" : {  
"uuid" : "q3imiZGQTDasQUuMWS8oiw",  
"number\_of\_replicas" : "1",  
"number\_of\_shards" : "6",  
"refresh\_interval" : "600s",  
"version" : {  
"created" : "1020199"  
}  
}  
}  
}  
}

Create index, setting refresh\_interval and load is done on that same  
cluster node. Before test index is deleted and created again before start  
new test with new value of refresh\_interval. All cluster nodes logs  
information that parameter has been changed, eg:  
[2014-07-16 11:24:09,813][INFO][index.shard.service] [h6]  
[smt\_20140501\_100000\_20g\_norefresh][1] updating refresh\_interval from [1s]  
to [-1]  
or  
[2014-07-16 11:32:32,928][INFO][index.shard.service] [h6]  
[smt\_20140501\_100000\_20g\_norefresh][1] updating refresh\_interval from [1s]  
to [10m]

After start test new data are available immediately and indexing time that  
same in 3 cases. I don't know where is failure. Somebody know what is going  
on?

Regards  
Marek

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![mikemccand](https://avatars.discourse-cdn.com/v4/letter/m/f04885/32.png) [@mikemccand](https://discuss.elastic.co/u/mikemccand)
#### Post date: [July 16, 2014, 2:52pm UTC](https://discuss.elastic.co/t/no-efect-refresh-interval/18694/2 "2014-07-16T14:52:00Z")

</div>

Which ES version are you using? You should use the latest (soon to be  
1.3): there have been a number of bulk-indexing improvements recently.

Are you using the bulk API with multiple/async client threads? Are you  
saturating either CPU or IO in your cluster (so that the test is really a  
full cluster capacity test)?

Also, the relationship between refresh\_interval and indexing performance is  
tricky: it turns out, -1 is often a poor choice, because it means your bulk  
indexing threads are sometimes tied up flushing segments when with  
refreshing enabled, it's a separate thread that does that. So a refresh of  
5s is maybe a good choice.

Mike McCandless

[http://blog.mikemccandless.com](http://blog.mikemccandless.com)

On Wed, Jul 16, 2014 at 6:51 AM, Marek Dabrowski [marek.dabrowski@gmail.com](mailto:marek.dabrowski@gmail.com)  
wrote:

> Hello
> 
> My configuration is:  
> 6 nodes Elasticsearch cluster  
> OS: Centos 6.5  
> JVM: 1.7.0\_25
> 
> Cluster is working fine. I can indexing data, query, etc. Now I'm doing  
> test on package about ~50mln doc (~13GB). I would like take better  
> performance during indexing data. To take this target I has been changed  
> parameter refresh\_interval. I did test for 1s, -1 and 600s. Time for  
> indexing data is that same. I checked configuration (\_settings) for index  
> and value for refresh\_interval is ok (has proper value), eg:
> 
> {  
> "smt\_20140501\_100000\_20g\_norefresh" : {  
> "settings" : {  
> "index" : {  
> "uuid" : "q3imiZGQTDasQUuMWS8oiw",  
> "number\_of\_replicas" : "1",  
> "number\_of\_shards" : "6",  
> "refresh\_interval" : "600s",  
> "version" : {  
> "created" : "1020199"  
> }  
> }  
> }  
> }  
> }
> 
> Create index, setting refresh\_interval and load is done on that same  
> cluster node. Before test index is deleted and created again before start  
> new test with new value of refresh\_interval. All cluster nodes logs  
> information that parameter has been changed, eg:  
> [2014-07-16 11:24:09,813][INFO][index.shard.service] [h6]  
> [smt\_20140501\_100000\_20g\_norefresh][1] updating refresh\_interval from [1s]  
> to [-1]  
> or  
> [2014-07-16 11:32:32,928][INFO][index.shard.service] [h6]  
> [smt\_20140501\_100000\_20g\_norefresh][1] updating refresh\_interval from [1s]  
> to [10m]
> 
> After start test new data are available immediately and indexing time that  
> same in 3 cases. I don't know where is failure. Somebody know what is going  
> on?
> 
> Regards  
> Marek
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAD7smRezWeZFQMSMVXj7ELW0xGSu3sPRxfXqcuF4bmtrLVBjYg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAD7smRezWeZFQMSMVXj7ELW0xGSu3sPRxfXqcuF4bmtrLVBjYg%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![Marek\_Dabrowski](https://avatars.discourse-cdn.com/v4/letter/m/b5ac83/32.png) [@Marek\_Dabrowski](https://discuss.elastic.co/u/Marek_Dabrowski)
#### Post date: [July 17, 2014, 7:59am UTC](https://discuss.elastic.co/t/no-efect-refresh-interval/18694/3 "2014-07-17T07:59:55Z")

</div>

Hello Mike

My ES version is 1.2.1  
I checked utilization nodes my cluster. Common valus ofr all nodes are:  
java proces cpu utilization: \< 6%  
os load: \< 1  
io stat: \< 15kB/s write

I checked indexing process 2 methods:  
a) indexing by native json data (13GB splited to 100MB chunks)  
time for i in /tmp/SMT\* ; do echo $i; curl -s -XPOST  
h3:9200/smt\_20140501\_bulk\_json\_refresh\_600/num/\_bulk --data-binary @$i ; rm  
-f $i; done

b) indexing csv data by use perl script

my $e = Search::Elasticsearch-\>new(  
nodes =\> [  
'h3:9200',  
]  
);

my $bulk = $e-\>bulk\_helper(  
index =\> $idx\_name,  
type =\> $idx\_type,  
max\_count =\> 10000  
);

open(my $DATA, '\<', $data\_file) or die $!;  
while(\<$DATA\>) {  
chomp;

```
my @data = split(',', $_);
$bulk->index({ source => {  
                            p0 => $data[0], 
                            p1 => $data[1],
                            p2 => $data[2],
                            p3 => $data[3],
                            p4 => $data[4],
                            p5 => $data[5],
                            p6 => $data[6],
                            p7 => $data[7],
                            p8 => $data[8],
                            p9 => $data[9],
                            p10 => $data[10],
                            p11 => $data[11]
            }});

```

}  
close($DATA);  
$bulk-\>flush;

Setting refresh\_interval to 600s in both cases has no effect. Data are  
available immediately. I expect (equal to ES documentation) that new data  
will be available after 10 minutes and in consequently indexing process  
will be quicker but it doesn’t.

Regards

W dniu środa, 16 lipca 2014 16:52:31 UTC+2 użytkownik Michael McCandless  
napisał:

> Which ES version are you using? You should use the latest (soon to be  
> 1.3): there have been a number of bulk-indexing improvements recently.
> 
> Are you using the bulk API with multiple/async client threads? Are you  
> saturating either CPU or IO in your cluster (so that the test is really a  
> full cluster capacity test)?
> 
> Also, the relationship between refresh\_interval and indexing performance  
> is tricky: it turns out, -1 is often a poor choice, because it means your  
> bulk indexing threads are sometimes tied up flushing segments when with  
> refreshing enabled, it's a separate thread that does that. So a refresh of  
> 5s is maybe a good choice.
> 
> Mike McCandless
> 
> [http://blog.mikemccandless.com](http://blog.mikemccandless.com)
> 
> On Wed, Jul 16, 2014 at 6:51 AM, Marek Dabrowski \<[marek.d...@gmail.com](mailto:marek.d...@gmail.com)  
> \<javascript:\>\> wrote:
> 
> > Hello
> > 
> > My configuration is:  
> > 6 nodes Elasticsearch cluster  
> > OS: Centos 6.5  
> > JVM: 1.7.0\_25
> > 
> > Cluster is working fine. I can indexing data, query, etc. Now I'm doing  
> > test on package about ~50mln doc (~13GB). I would like take better  
> > performance during indexing data. To take this target I has been changed  
> > parameter refresh\_interval. I did test for 1s, -1 and 600s. Time for  
> > indexing data is that same. I checked configuration (\_settings) for index  
> > and value for refresh\_interval is ok (has proper value), eg:
> > 
> > {  
> > "smt\_20140501\_100000\_20g\_norefresh" : {  
> > "settings" : {  
> > "index" : {  
> > "uuid" : "q3imiZGQTDasQUuMWS8oiw",  
> > "number\_of\_replicas" : "1",  
> > "number\_of\_shards" : "6",  
> > "refresh\_interval" : "600s",  
> > "version" : {  
> > "created" : "1020199"  
> > }  
> > }  
> > }  
> > }  
> > }
> > 
> > Create index, setting refresh\_interval and load is done on that same  
> > cluster node. Before test index is deleted and created again before start  
> > new test with new value of refresh\_interval. All cluster nodes logs  
> > information that parameter has been changed, eg:  
> > [2014-07-16 11:24:09,813][INFO][index.shard.service] [h6]  
> > [smt\_20140501\_100000\_20g\_norefresh][1] updating refresh\_interval from [1s]  
> > to [-1]  
> > or  
> > [2014-07-16 11:32:32,928][INFO][index.shard.service] [h6]  
> > [smt\_20140501\_100000\_20g\_norefresh][1] updating refresh\_interval from [1s]  
> > to [10m]
> > 
> > After start test new data are available immediately and indexing time  
> > that same in 3 cases. I don't know where is failure. Somebody know what is  
> > going on?
> > 
> > Regards  
> > Marek
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .  
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7c6e2fef-4e40-44d8-a1ea-eade7880d5d5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7c6e2fef-4e40-44d8-a1ea-eade7880d5d5%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![Marek\_Dabrowski](https://avatars.discourse-cdn.com/v4/letter/m/b5ac83/32.png) [@Marek\_Dabrowski](https://discuss.elastic.co/u/Marek_Dabrowski)
#### Post date: [July 17, 2014, 1:02pm UTC](https://discuss.elastic.co/t/no-efect-refresh-interval/18694/4 "2014-07-17T13:02:28Z")

</div>

Hello

I found reason my problems.  
Refresh index during usage perl depend on parameters "max\_count" and  
"max\_size" for  
$e-\>bulk\_helper  
Values for this parameters determine when refresh will be done on index.

Tnx for help.

Regards

W dniu czwartek, 17 lipca 2014 09:59:55 UTC+2 użytkownik Marek Dabrowski  
napisał:

> Hello Mike
> 
> My ES version is 1.2.1  
> I checked utilization nodes my cluster. Common valus ofr all nodes are:  
> java proces cpu utilization: \< 6%  
> os load: \< 1  
> io stat: \< 15kB/s write
> 
> I checked indexing process 2 methods:  
> a) indexing by native json data (13GB splited to 100MB chunks)  
> time for i in /tmp/SMT\* ; do echo $i; curl -s -XPOST  
> h3:9200/smt\_20140501\_bulk\_json\_refresh\_600/num/\_bulk --data-binary @$i ; rm  
> -f $i; done
> 
> b) indexing csv data by use perl script
> 
> my $e = Search::Elasticsearch-\>new(  
> nodes =\> [  
> 'h3:9200',  
> ]  
> );
> 
> my $bulk = $e-\>bulk\_helper(  
> index =\> $idx\_name,  
> type =\> $idx\_type,  
> max\_count =\> 10000  
> );
> 
> open(my $DATA, '\<', $data\_file) or die $!;  
> while(\<$DATA\>) {  
> chomp;
> 
> ```
> my @data = split(',', $_);
> $bulk->index({ source => {  
> p0 => $data[0], 
> p1 => $data[1],
> p2 => $data[2],
> p3 => $data[3],
> p4 => $data[4],
> p5 => $data[5],
> p6 => $data[6],
> p7 => $data[7],
> p8 => $data[8],
> p9 => $data[9],
> p10 => $data[10],
> p11 => $data[11]
> }});
> 
> ```
> 
> }  
> close($DATA);  
> $bulk-\>flush;
> 
> Setting refresh\_interval to 600s in both cases has no effect. Data are  
> available immediately. I expect (equal to ES documentation) that new data  
> will be available after 10 minutes and in consequently indexing process  
> will be quicker but it doesn’t.
> 
> Regards
> 
> W dniu środa, 16 lipca 2014 16:52:31 UTC+2 użytkownik Michael McCandless  
> napisał:
> 
> > Which ES version are you using? You should use the latest (soon to be  
> > 1.3): there have been a number of bulk-indexing improvements recently.
> > 
> > Are you using the bulk API with multiple/async client threads? Are you  
> > saturating either CPU or IO in your cluster (so that the test is really a  
> > full cluster capacity test)?
> > 
> > Also, the relationship between refresh\_interval and indexing performance  
> > is tricky: it turns out, -1 is often a poor choice, because it means your  
> > bulk indexing threads are sometimes tied up flushing segments when with  
> > refreshing enabled, it's a separate thread that does that. So a refresh of  
> > 5s is maybe a good choice.
> > 
> > Mike McCandless
> > 
> > [http://blog.mikemccandless.com](http://blog.mikemccandless.com)
> > 
> > On Wed, Jul 16, 2014 at 6:51 AM, Marek Dabrowski [marek.d...@gmail.com](mailto:marek.d...@gmail.com)  
> > wrote:
> > 
> > > Hello
> > > 
> > > My configuration is:  
> > > 6 nodes Elasticsearch cluster  
> > > OS: Centos 6.5  
> > > JVM: 1.7.0\_25
> > > 
> > > Cluster is working fine. I can indexing data, query, etc. Now I'm doing  
> > > test on package about ~50mln doc (~13GB). I would like take better  
> > > performance during indexing data. To take this target I has been changed  
> > > parameter refresh\_interval. I did test for 1s, -1 and 600s. Time for  
> > > indexing data is that same. I checked configuration (\_settings) for index  
> > > and value for refresh\_interval is ok (has proper value), eg:
> > > 
> > > {  
> > > "smt\_20140501\_100000\_20g\_norefresh" : {  
> > > "settings" : {  
> > > "index" : {  
> > > "uuid" : "q3imiZGQTDasQUuMWS8oiw",  
> > > "number\_of\_replicas" : "1",  
> > > "number\_of\_shards" : "6",  
> > > "refresh\_interval" : "600s",  
> > > "version" : {  
> > > "created" : "1020199"  
> > > }  
> > > }  
> > > }  
> > > }  
> > > }
> > > 
> > > Create index, setting refresh\_interval and load is done on that same  
> > > cluster node. Before test index is deleted and created again before start  
> > > new test with new value of refresh\_interval. All cluster nodes logs  
> > > information that parameter has been changed, eg:  
> > > [2014-07-16 11:24:09,813][INFO][index.shard.service] [h6]  
> > > [smt\_20140501\_100000\_20g\_norefresh][1] updating refresh\_interval from [1s]  
> > > to [-1]  
> > > or  
> > > [2014-07-16 11:32:32,928][INFO][index.shard.service] [h6]  
> > > [smt\_20140501\_100000\_20g\_norefresh][1] updating refresh\_interval from [1s]  
> > > to [10m]
> > > 
> > > After start test new data are available immediately and indexing time  
> > > that same in 3 cases. I don't know where is failure. Somebody know what is  
> > > going on?
> > > 
> > > Regards  
> > > Marek
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google  
> > > Groups "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send  
> > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > > To view this discussion on the web visit  
> > > [https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com)  
> > > [https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > > .  
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/fb81cccc-d826-46d0-b37f-ca63e74093d2%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/fb81cccc-d826-46d0-b37f-ca63e74093d2%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![mikemccand](https://avatars.discourse-cdn.com/v4/letter/m/f04885/32.png) [@mikemccand](https://discuss.elastic.co/u/mikemccand)
#### Post date: [July 17, 2014, 1:28pm UTC](https://discuss.elastic.co/t/no-efect-refresh-interval/18694/5 "2014-07-17T13:28:25Z")

</div>

OK, thanks for bringing closure.

Mike McCandless

[http://blog.mikemccandless.com](http://blog.mikemccandless.com)

On Thu, Jul 17, 2014 at 9:02 AM, Marek Dabrowski [marek.dabrowski@gmail.com](mailto:marek.dabrowski@gmail.com)  
wrote:

> Hello
> 
> I found reason my problems.  
> Refresh index during usage perl depend on parameters "max\_count" and  
> "max\_size" for  
> $e-\>bulk\_helper  
> Values for this parameters determine when refresh will be done on index.
> 
> Tnx for help.
> 
> Regards
> 
> W dniu czwartek, 17 lipca 2014 09:59:55 UTC+2 użytkownik Marek Dabrowski  
> napisał:
> 
> > Hello Mike
> > 
> > My ES version is 1.2.1  
> > I checked utilization nodes my cluster. Common valus ofr all nodes are:  
> > java proces cpu utilization: \< 6%  
> > os load: \< 1  
> > io stat: \< 15kB/s write
> > 
> > I checked indexing process 2 methods:  
> > a) indexing by native json data (13GB splited to 100MB chunks)  
> > time for i in /tmp/SMT\* ; do echo $i; curl -s -XPOST  
> > h3:9200/smt\_20140501\_bulk\_json\_refresh\_600/num/\_bulk --data-binary @$i ;  
> > rm -f $i; done
> > 
> > b) indexing csv data by use perl script
> > 
> > my $e = Search::Elasticsearch-\>new(  
> > nodes =\> [  
> > 'h3:9200',  
> > ]  
> > );
> > 
> > my $bulk = $e-\>bulk\_helper(  
> > index =\> $idx\_name,  
> > type =\> $idx\_type,  
> > max\_count =\> 10000  
> > );
> > 
> > open(my $DATA, '\<', $data\_file) or die $!;  
> > while(\<$DATA\>) {  
> > chomp;
> > 
> > ```
> > my @data = split(',', $_);
> > $bulk->index({ source => {
> > p0 => $data[0],
> > p1 => $data[1],
> > p2 => $data[2],
> > p3 => $data[3],
> > p4 => $data[4],
> > p5 => $data[5],
> > p6 => $data[6],
> > p7 => $data[7],
> > p8 => $data[8],
> > p9 => $data[9],
> > p10 => $data[10],
> > p11 => $data[11]
> > }});
> > 
> > ```
> > 
> > }  
> > close($DATA);  
> > $bulk-\>flush;
> > 
> > Setting refresh\_interval to 600s in both cases has no effect. Data are  
> > available immediately. I expect (equal to ES documentation) that new data  
> > will be available after 10 minutes and in consequently indexing process  
> > will be quicker but it doesn’t.
> > 
> > Regards
> > 
> > W dniu środa, 16 lipca 2014 16:52:31 UTC+2 użytkownik Michael McCandless  
> > napisał:
> > 
> > > Which ES version are you using? You should use the latest (soon to be  
> > > 1.3): there have been a number of bulk-indexing improvements recently.
> > > 
> > > Are you using the bulk API with multiple/async client threads? Are you  
> > > saturating either CPU or IO in your cluster (so that the test is really a  
> > > full cluster capacity test)?
> > > 
> > > Also, the relationship between refresh\_interval and indexing performance  
> > > is tricky: it turns out, -1 is often a poor choice, because it means your  
> > > bulk indexing threads are sometimes tied up flushing segments when with  
> > > refreshing enabled, it's a separate thread that does that. So a refresh of  
> > > 5s is maybe a good choice.
> > > 
> > > Mike McCandless
> > > 
> > > [http://blog.mikemccandless.com](http://blog.mikemccandless.com)
> > > 
> > > On Wed, Jul 16, 2014 at 6:51 AM, Marek Dabrowski [marek.d...@gmail.com](mailto:marek.d...@gmail.com)  
> > > wrote:
> > > 
> > > > Hello
> > > > 
> > > > My configuration is:  
> > > > 6 nodes Elasticsearch cluster  
> > > > OS: Centos 6.5  
> > > > JVM: 1.7.0\_25
> > > > 
> > > > Cluster is working fine. I can indexing data, query, etc. Now I'm doing  
> > > > test on package about ~50mln doc (~13GB). I would like take better  
> > > > performance during indexing data. To take this target I has been changed  
> > > > parameter refresh\_interval. I did test for 1s, -1 and 600s. Time for  
> > > > indexing data is that same. I checked configuration (\_settings) for index  
> > > > and value for refresh\_interval is ok (has proper value), eg:
> > > > 
> > > > {  
> > > > "smt\_20140501\_100000\_20g\_norefresh" : {  
> > > > "settings" : {  
> > > > "index" : {  
> > > > "uuid" : "q3imiZGQTDasQUuMWS8oiw",  
> > > > "number\_of\_replicas" : "1",  
> > > > "number\_of\_shards" : "6",  
> > > > "refresh\_interval" : "600s",  
> > > > "version" : {  
> > > > "created" : "1020199"  
> > > > }  
> > > > }  
> > > > }  
> > > > }  
> > > > }
> > > > 
> > > > Create index, setting refresh\_interval and load is done on that same  
> > > > cluster node. Before test index is deleted and created again before start  
> > > > new test with new value of refresh\_interval. All cluster nodes logs  
> > > > information that parameter has been changed, eg:  
> > > > [2014-07-16 11:24:09,813][INFO][index.shard.service] [h6]  
> > > > [smt\_20140501\_100000\_20g\_norefresh][1] updating refresh\_interval from  
> > > > [1s] to [-1]  
> > > > or  
> > > > [2014-07-16 11:32:32,928][INFO][index.shard.service] [h6]  
> > > > [smt\_20140501\_100000\_20g\_norefresh][1] updating refresh\_interval from  
> > > > [1s] to [10m]
> > > > 
> > > > After start test new data are available immediately and indexing time  
> > > > that same in 3 cases. I don't know where is failure. Somebody know what is  
> > > > going on?
> > > > 
> > > > Regards  
> > > > Marek
> > > > 
> > > > --  
> > > > You received this message because you are subscribed to the Google  
> > > > Groups "elasticsearch" group.  
> > > > To unsubscribe from this group and stop receiving emails from it, send  
> > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/)  
> > > > msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%  
> > > > [40googlegroups.com](http://40googlegroups.com)  
> > > > [https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f7565c36-98c7-4e3e-8132-796f9edfb3fa%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > > > .  
> > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google Groups  
> > > "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send an  
> > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > > To view this discussion on the web visit  
> > > [https://groups.google.com/d/msgid/elasticsearch/fb81cccc-d826-46d0-b37f-ca63e74093d2%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/fb81cccc-d826-46d0-b37f-ca63e74093d2%40googlegroups.com)  
> > > [https://groups.google.com/d/msgid/elasticsearch/fb81cccc-d826-46d0-b37f-ca63e74093d2%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/fb81cccc-d826-46d0-b37f-ca63e74093d2%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > > .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAD7smRc85-8vuZdNzwYP4mbsba7SBDHA2whdGuyaj0%2BLLG\_\_hQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAD7smRc85-8vuZdNzwYP4mbsba7SBDHA2whdGuyaj0%2BLLG__hQ%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:15am UTC](https://discuss.elastic.co/t/no-efect-refresh-interval/18694/6 "2017-07-06T01:15:13Z")

</div>


