# Optimum index.translog.flush\_threshold\_ops setting **URL:** \ **Category:** Elasticsearch\ **Created:** [May 4, 2013, 5:03am UTC](https://discuss.elastic.co/t/optimum-index-translog-flush-threshold-ops-setting/11805 "2013-05-04T05:03:17Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![subin](https://avatars.discourse-cdn.com/v4/letter/s/58956e/32.png) [@subin](https://discuss.elastic.co/u/subin)\ **Post date:** [May 4, 2013, 5:03am UTC](https://discuss.elastic.co/t/optimum-index-translog-flush-threshold-ops-setting/11805/1 "2013-05-04T05:03:17Z")
I'm not sure how to deduce the optimum "index.translog.flush\_threshold\_ops" for my Logstash setup. Logstash indexes are created on daily basis. I'd like some advises. I hope these helps: i. ES version- 0.90.0 stable ii. ES index template curl -XPUT [http://10.0.4.24:9200/\_template/loggerstash](http://10.0.4.24:9200/_template/loggerstash) -d ' { "template" : "logstash\*", "settings" : { "number\_of\_shards" : 1, "index.cache.field.type" : "soft", "index.refresh\_interval" : "10s", "index.store.compress.stored" : true } }' iii. Logstash pushes approx. 2800 logs/sec into ES. iv. ES is tuned with ES\_HEAP\_SIZE=3g MAX\_LOCKED\_MEMORY=unlimited MAX\_OPEN\_FILES=65535 |||bootstrap.mlockall: ||true Do I need further change other parameters mentioned here [http://www.elasticsearch.org/guide/reference/index-modules/translog/](http://www.elasticsearch.org/guide/reference/index-modules/translog/) other than the default? Thanks, \_\_| -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![radu\_gheorghe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radu_gheorghe/32/556_2.png) [@radu\_gheorghe](https://discuss.elastic.co/u/radu_gheorghe)\ **Post date:** [May 4, 2013, 8:10am UTC](https://discuss.elastic.co/t/optimum-index-translog-flush-threshold-ops-setting/11805/2 "2013-05-04T08:10:23Z")
Hello, I think tuning the flush threshold with 2.8k logs/s is like tuning your bulk size: increase it, then see if you get a significant performance gain. If you do, increase it again until you don't get a significant gain. The default setting of 5000 means you'll have a flush every 2 seconds or even less. I'd make it 10 times more than that. Assuming your logs are fairly small (under 1K or so), it shouldn't make up a huge transaction log. Some other advice regarding your template: - in my experience, having 1 shard per index doesn't help you significantly in terms of search performance. Having 5 shards instead of 1, however, should help boost the indexing performance, and also gives you room for adding more nodes to host the same index - soft field caches will put pressure on your CPU because of the Garbage Collector. And CPU is a precious resource when indexing. Since you're on 0.90, I suggest you cap the size of it by setting index.fielddata.cache.size to something like 20% or whatever fits your needs in terms of search performance vs used memory - increasing indices.memory.index\_buffer\_size from the default 10% might help your indexing speed. As with the translog, I think it's a matter of trial-and-error to get the right size - in 0.90, you get compression always enabled at Lucene-level, so "index.store.compress.stored" : true shouldn't have any effect ## Best regards, Radu [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene On Sat, May 4, 2013 at 8:03 AM, Subin [ksubins321@gmail.com](mailto:ksubins321@gmail.com) wrote: > I'm not sure how to deduce the optimum > "index.translog.flush\_threshold\_ops" for my Logstash setup. > Logstash indexes are created on daily basis. I'd like some advises. I hope > these helps: > > i. ES version- 0.90.0 stable > ii. ES index template > curl -XPUT [http://10.0.4.24:9200/\_template/loggerstash](http://10.0.4.24:9200/_template/loggerstash) -d ' > { > > ``` > "template" : > > ``` > > "logstash\*", > "settings" : { "number\_of\_shards" : 1, > "index.cache.field.type" : "soft", > "index.refresh\_interval" : "10s", > "index.store.compress.stored" : true } > }' > iii. Logstash pushes approx. 2800 logs/sec into ES. > iv. ES is tuned with > ES\_HEAP\_SIZE=3g > MAX\_LOCKED\_MEMORY=unlimited > MAX\_OPEN\_FILES=65535 > bootstrap.mlockall: true > > Do I need further change other parameters mentioned here[http://www.elasticsearch.org/guide/reference/index-modules/translog/](http://www.elasticsearch.org/guide/reference/index-modules/translog/)other than the default? > > Thanks, > \*\* > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:38am UTC](https://discuss.elastic.co/t/optimum-index-translog-flush-threshold-ops-setting/11805/3 "2017-07-06T02:38:20Z")