When are index updates written to the gateway?

Elastic Stack Elasticsearch
1

In our system we would like to know an index submission has been fully
written to the gateway before acknowledging a document has been
indexed.

The default configuration is for index commits to occur every second.
In our configuration, we're calling snapshot programatically. Does
snapshot force an index commit or does it work from the last commit?

Thanks,

David

2

elasticsearch works a bit differently. You don't need a commit to make sure
data has been indexed in a persistent manner. There is an internal
transaction log that makes sure this is the case. Once an index/delete
operation returns, it is after it has been indexed and added to the
transaction log (no need for lucene commit, which is expensive).

Regarding the shared gateway, it periodically (and in an asyn manner)
replicas those changes to a shared storage. This happens every 10 seconds
(by default), regardless of commits / flush or not (transaction log is also
persisted to the shared storage).

You can have a look at the local gateway option added in upcoming 0.11, it
basically provides full long term persistency without a shared storage
gateway. Blogged about it here:
Zero Conf Persistency | Elastic Blog.

-shay.banon

On Mon, Sep 27, 2010 at 4:27 PM, dbenson dbenson@dbenson.net wrote:

In our system we would like to know an index submission has been fully
written to the gateway before acknowledging a document has been
indexed.

The default configuration is for index commits to occur every second.
In our configuration, we're calling snapshot programatically. Does
snapshot force an index commit or does it work from the last commit?

Thanks,

David