It basically boils down to how lucene works when storing the index, and the
additional md5 checksum files elasticsearch produces for them. Basically, an
index "version" is written to the gateway while another one exists, and only
when its done being written to the gateway, then the "old" one is removed.
The transaction log is an append only log, and keyed by the index version,
so you just copy it over, and how many operations managed to get into it,
you will get when you recovery.
On Wed, Aug 4, 2010 at 11:05 PM, Lukáš Vlček firstname.lastname@example.org wrote:
do you think you can elaborate more on this? I am surprised this is
possible. My naive understanding is that if the snapshotting is going on
then some files in gateway are changed, how it is then possible that the
copy is consistent?
On Wed, Aug 4, 2010 at 9:50 PM, Shay Banon email@example.com:
Yes, you can safely copy over the gateway data, either to another s3 or
filesystem, even while its snapshotting.
On Wed, Aug 4, 2010 at 10:46 PM, Grant Rodgers firstname.lastname@example.org wrote:
We would like to copy the gateway snapshot from a production cluster
to a development cluster. Since our production gateway is on s3,
copying indices to development could take several minutes, during
which the production cluster would still be snapshotting. Is this
safe? If not, could we disable snapshotting temporarily while the copy
Also, is it possible to copy an s3 gateway to a filesystem gateway?
We are running ES 0.9.