# Update durability vs Solr **URL:** \ **Category:** Elasticsearch\ **Created:** [July 16, 2012, 4:33pm UTC](https://discuss.elastic.co/t/update-durability-vs-solr/8427 "2012-07-16T16:33:21Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Michael\_McCandless](https://avatars.discourse-cdn.com/v4/letter/m/e47c2d/32.png) [@Michael\_McCandless](https://discuss.elastic.co/u/Michael_McCandless)\ **Post date:** [July 16, 2012, 4:33pm UTC](https://discuss.elastic.co/t/update-durability-vs-solr/8427/1 "2012-07-16T16:33:21Z")
What is the default update durability with ElasticSearch...? Meaning, on OS crash or power loss of one or more (or all) nodes in a cluster, will _all_ updates be replayed on the next start up? (Is the transaction log sync'd after every update? That seems incredibly costly...). Peeking at the sources it looks like there is a buffering option (add to the log after N changes/MB have buffered up in RAM) and a non-buffering option ... so I'm wondering what the default is. It looks like (not sure though) Solr does not sync the transaction log after every update (see [http://lucene.markmail.org/thread/qfpv7sq5pe4tuwq6](http://lucene.markmail.org/thread/qfpv7sq5pe4tuwq6) ) so I'm trying to compare the two... Thanks. Mike [http://blog.mikemccandless.com](http://blog.mikemccandless.com) ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [July 16, 2012, 5:21pm UTC](https://discuss.elastic.co/t/update-durability-vs-solr/8427/2 "2012-07-16T17:21:39Z")
Hi Mike > Peeking at the sources it looks like there is a buffering option (add > to the log after N changes/MB have buffered up in RAM) and a > non-buffering option ... so I'm wondering what the default is. It looks like the default is 64kB: > > > Buffer writes when not sync'ing on each operation. Sync will cause flush and syn…c. Buffering will be on by default if not sync'ing on each operation. Settings (can be dynamically updated using update settings API): > \- \`index.translog.fs.type\`: Either \`simple\` or \`buffered\`. Defaults to \`buffer\` if not sync'ing on each operation. > \- \`index.translog.fs.buffer\_size\`: The buffer size, defaults to \`64k\`. clint ---
**Author:** ![Michael\_McCandless](https://avatars.discourse-cdn.com/v4/letter/m/e47c2d/32.png) [@Michael\_McCandless](https://discuss.elastic.co/u/Michael_McCandless)\ **Post date:** [July 16, 2012, 7:42pm UTC](https://discuss.elastic.co/t/update-durability-vs-solr/8427/3 "2012-07-16T19:42:12Z")
On Mon, Jul 16, 2012 at 1:21 PM, Clinton Gormley [clint@traveljury.com](mailto:clint@traveljury.com) wrote: > Hi Mike > > > Peeking at the sources it looks like there is a buffering option (add > > to the log after N changes/MB have buffered up in RAM) and a > > non-buffering option ... so I'm wondering what the default is. > > It looks like the default is 64kB: > [Translog: When not sync'ing on each operation, buffer writes · Issue #1549 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1549) OK, thanks Clinton. So the default buffer size when not sync'ing on each update is 64 KB ... does that mean after 64 KB it's flushed and sync'd? So at worse you'd lose the last 64 KB updates if the cluster goes down? Separately: is buffering (not sync'ing on each update) the default? Mike [http://blog.mikemccandless.com](http://blog.mikemccandless.com) ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 17, 2012, 7:45pm UTC](https://discuss.elastic.co/t/update-durability-vs-solr/8427/4 "2012-07-17T19:45:56Z")
Hi Mike, the default setting of the translog buffering depends on the setting of the parameter index.gateway.local.sync (which defaults to "5s") index.gateway.local.sync \> 0: the translog is not sync'd on each operation but after n milliseconds, so buffering is used index.gateway.local.sync = 0: the translog is sync'd on each operation, buffering is disabled index.gateway.local.sync \< 0: the translog is never sync'd Best regards, Jörg ---
**Author:** ![Michael\_McCandless](https://avatars.discourse-cdn.com/v4/letter/m/e47c2d/32.png) [@Michael\_McCandless](https://discuss.elastic.co/u/Michael_McCandless)\ **Post date:** [July 17, 2012, 8:03pm UTC](https://discuss.elastic.co/t/update-durability-vs-solr/8427/5 "2012-07-17T20:03:28Z")
Awesome thanks Jörg: that answers my question. So changes committed earlier than 5s ago should not be lost if cluster goes down, by default. When comparing indexing throughput against Solr (or other search apps) it's important to match the translog configuration, if possible, because sync is so costly. Mike [http://blog.mikemccandless.com](http://blog.mikemccandless.com) On Tue, Jul 17, 2012 at 3:45 PM, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Hi Mike, > > the default setting of the translog buffering depends on the setting of the > parameter index.gateway.local.sync (which defaults to "5s") > > index.gateway.local.sync \> 0: the translog is not sync'd on each operation > but after n milliseconds, so buffering is used > index.gateway.local.sync = 0: the translog is sync'd on each operation, > buffering is disabled > index.gateway.local.sync \< 0: the translog is never sync'd > > Best regards, > > Jörg ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [July 18, 2012, 8:48pm UTC](https://discuss.elastic.co/t/update-durability-vs-solr/8427/6 "2012-07-18T20:48:37Z")
Note also, that elasticsearch does (by default) sync replication to all shard copies of the data, each including its own transaction log. On Tue, Jul 17, 2012 at 1:03 PM, Michael McCandless \<[mail@mikemccandless.com](mailto:mail@mikemccandless.com) > wrote: > Awesome thanks Jörg: that answers my question. > > So changes committed earlier than 5s ago should not be lost if cluster > goes down, by default. > > When comparing indexing throughput against Solr (or other search apps) > it's important to match the translog configuration, if possible, > because sync is so costly. > > Mike > > [http://blog.mikemccandless.com](http://blog.mikemccandless.com) > > On Tue, Jul 17, 2012 at 3:45 PM, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com) > wrote: > > > Hi Mike, > > > > the default setting of the translog buffering depends on the setting of > > the > > parameter index.gateway.local.sync (which defaults to "5s") > > > > index.gateway.local.sync \> 0: the translog is not sync'd on each > > operation > > but after n milliseconds, so buffering is used > > index.gateway.local.sync = 0: the translog is sync'd on each operation, > > buffering is disabled > > index.gateway.local.sync \< 0: the translog is never sync'd > > > > Best regards, > > > > Jörg ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:19am UTC](https://discuss.elastic.co/t/update-durability-vs-solr/8427/7 "2017-07-06T03:19:53Z")