I've read various information about the concept of Gateway in ES and
I'm wondering what value Gateway adds when one is not deploying ES in
a cloud? In a cloud like AWS local storage is not necessarily
persistent - when you stop an EC2 instance I believe its local data is
gone. But what if you are deploying ES to a set of your own servers
where this is not the case? Which type of gw should one use? local?
fs?
I've read various information about the concept of Gateway in ES and
I'm wondering what value Gateway adds when one is not deploying ES in
a cloud? In a cloud like AWS local storage is not necessarily
persistent - when you stop an EC2 instance I believe its local data is
gone. But what if you are deploying ES to a set of your own servers
where this is not the case? Which type of gw should one use? local?
fs?
In case of "local" all index data stays local spread over N replicas
and M shards.
And the index settings/configs/mappings/state/etc. is all stored
locally (on local disks), right? And what about replicas of this
data? Is this stored in 1 central place or are there copies/replicas
of index settings/configs/mappings/state/etc. on multiple cluster
nodes? I couldn't find this stated in the guide or anywhere else.
I've read various information about the concept of Gateway in ES and
I'm wondering what value Gateway adds when one is not deploying ES in
a cloud? In a cloud like AWS local storage is not necessarily
persistent - when you stop an EC2 instance I believe its local data is
gone. But what if you are deploying ES to a set of your own servers
where this is not the case? Which type of gw should one use? local?
fs?
In case of "local" all index data stays local spread over N replicas
and M shards.
And the index settings/configs/mappings/state/etc. is all stored
locally (on local disks), right? And what about replicas of this
data? Is this stored in 1 central place or are there copies/replicas
of index settings/configs/mappings/state/etc. on multiple cluster
nodes? I couldn't find this stated in the guide or anywhere else.
I've read various information about the concept of Gateway in ES and
I'm wondering what value Gateway adds when one is not deploying ES in
a cloud? In a cloud like AWS local storage is not necessarily
persistent - when you stop an EC2 instance I believe its local data is
gone. But what if you are deploying ES to a set of your own servers
where this is not the case? Which type of gw should one use? local?
fs?
Would it then be true to say that pretty much the only (or at least
the main) scenario where Shared FS gw makes more sense over the Local
gw is when one is dealing with servers whose local storage is
ephemeral/temporary, as is the case in EC2?
In case of "local" all index data stays local spread over N replicas
and M shards.
And the index settings/configs/mappings/state/etc. is all stored
locally (on local disks), right? And what about replicas of this
data? Is this stored in 1 central place or are there copies/replicas
of index settings/configs/mappings/state/etc. on multiple cluster
nodes? I couldn't find this stated in the guide or anywhere else.
I've read various information about the concept of Gateway in ES and
I'm wondering what value Gateway adds when one is not deploying ES in
a cloud? In a cloud like AWS local storage is not necessarily
persistent - when you stop an EC2 instance I believe its local data is
gone. But what if you are deploying ES to a set of your own servers
where this is not the case? Which type of gw should one use? local?
fs?
Yes, that is the most common case. Another use case is for cases where an in
memory index is used, but long term persistency is still desirable in a
write behind fashion.
Note that even with the ec2 and local storage its not a clear cut. You can
still use local gateway with a high replica account, if you are ok with the
fact that if (number_of_replicas + 1) nodes are wiped out, you need to
reindex the data.
Would it then be true to say that pretty much the only (or at least
the main) scenario where Shared FS gw makes more sense over the Local
gw is when one is dealing with servers whose local storage is
ephemeral/temporary, as is the case in EC2?
In case of "local" all index data stays local spread over N replicas
and M shards.
And the index settings/configs/mappings/state/etc. is all stored
locally (on local disks), right? And what about replicas of this
data? Is this stored in 1 central place or are there copies/replicas
of index settings/configs/mappings/state/etc. on multiple cluster
nodes? I couldn't find this stated in the guide or anywhere else.
I've read various information about the concept of Gateway in ES
and
I'm wondering what value Gateway adds when one is not deploying ES
in
a cloud? In a cloud like AWS local storage is not necessarily
persistent - when you stop an EC2 instance I believe its local data
is
gone. But what if you are deploying ES to a set of your own
servers
where this is not the case? Which type of gw should one use?
local?
fs?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.