What is the default update durability with ElasticSearch...?
Meaning, on OS crash or power loss of one or more (or all) nodes in
a cluster, will all updates be replayed on the next start up? (Is the
transaction log sync'd after every update? That seems incredibly
costly...).
Peeking at the sources it looks like there is a buffering option (add
to the log after N changes/MB have buffered up in RAM) and a
non-buffering option ... so I'm wondering what the default is.
Peeking at the sources it looks like there is a buffering option (add
to the log after N changes/MB have buffered up in RAM) and a
non-buffering option ... so I'm wondering what the default is.
Peeking at the sources it looks like there is a buffering option (add
to the log after N changes/MB have buffered up in RAM) and a
non-buffering option ... so I'm wondering what the default is.
the default setting of the translog buffering depends on the setting of the
parameter index.gateway.local.sync (which defaults to "5s")
index.gateway.local.sync > 0: the translog is not sync'd on each operation
but after n milliseconds, so buffering is used
index.gateway.local.sync = 0: the translog is sync'd on each operation,
buffering is disabled
index.gateway.local.sync < 0: the translog is never sync'd
So changes committed earlier than 5s ago should not be lost if cluster
goes down, by default.
When comparing indexing throughput against Solr (or other search apps)
it's important to match the translog configuration, if possible,
because sync is so costly.
the default setting of the translog buffering depends on the setting of the
parameter index.gateway.local.sync (which defaults to "5s")
index.gateway.local.sync > 0: the translog is not sync'd on each operation
but after n milliseconds, so buffering is used
index.gateway.local.sync = 0: the translog is sync'd on each operation,
buffering is disabled
index.gateway.local.sync < 0: the translog is never sync'd
So changes committed earlier than 5s ago should not be lost if cluster
goes down, by default.
When comparing indexing throughput against Solr (or other search apps)
it's important to match the translog configuration, if possible,
because sync is so costly.
the default setting of the translog buffering depends on the setting of
the
parameter index.gateway.local.sync (which defaults to "5s")
index.gateway.local.sync > 0: the translog is not sync'd on each
operation
but after n milliseconds, so buffering is used
index.gateway.local.sync = 0: the translog is sync'd on each operation,
buffering is disabled
index.gateway.local.sync < 0: the translog is never sync'd
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.