# Just Pushed: Local Gateway
**URL:** https://discuss.elastic.co/t/just-pushed-local-gateway/3286
**Category:** Elasticsearch
**Created:** [August 31, 2010, 7:26pm UTC](https://discuss.elastic.co/t/just-pushed-local-gateway/3286 "2010-08-31T19:26:44Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [August 31, 2010, 7:26pm UTC](https://discuss.elastic.co/t/just-pushed-local-gateway/3286/1 "2010-08-31T19:26:44Z")
Hi,
```
I just pushed support for a `local` gateway:
```
[http://github.com/elasticsearch/elasticsearch/issues/issue/343](http://github.com/elasticsearch/elasticsearch/issues/issue/343). The idea of
the local gateway is to allow to perform full cluster recovery from local
information each node stores and not require a shared storage gateway (like
`fs` for shared file system, or `s3`, and `hdfs`).
```
The idea of a shared gateway works really well when local node data is
```
considered transient, but a full cluster recovery is still required. A
transient local data can be when deciding to store the index in memory.
Another benefit of using a shared storage gateway is to allow for easy
backup of the index. If a backup process is required (on top of the high
availability aspect of elasticsearch), it make sense to have it integrated
into how elasticsearch works, so the recovery process will take the backups
into account.
```
Still, there are many cases where reusing the local data stored on each
```
node when performing full recovery make sense. Very large indices for
example, that are good with N (\>1) replicas per shard, might find the shared
storage an overhead. Note that the shared storage model is not something
that other nosql solution provide, and most people are ok with relying on
local node storage and increased number of replicas. The good news is that
elasticsearch provides it now.
```
As a side note, shared storage model is something that takes some time
```
for people to understand (I can't count the number of times I have heard
something like: "ahh, it requires a shared storage, its not really
distributed then..."). Its a given when talking about "in memory" data
grids, since when bringing down the whole cluster, data was lost, and there
should be a way to recover it. With elasticsearch, this was my first
architecture decision for long term persistency, but I always had on the
roadmap the mentioned support.
```
Of course, this only make sense when using file based index storage (the
```
transaction log has moved to be file based since 0.8).
```
Enabling it is quite simple:
```
gateway:
type: "local"
```
Usually, you would want to configure the "gateway.recover_after_nodes"
```
setting to allow for more nodes to be in play when performing full cluster
restart so the correct cluster state is elected.
```
Last, I am considering make this setting the default one that comes with
```
the elasticsearch.yml setting. What do you think?
-shay.banon
---
### Author:  [@ppearcy](https://discuss.elastic.co/u/ppearcy)
#### Post date: [August 31, 2010, 7:35pm UTC](https://discuss.elastic.co/t/just-pushed-local-gateway/3286/2 "2010-08-31T19:35:11Z")
Hey Shay,
Cool stuff. Makes sense to me to be the default, but what ends up
happening when a shard that is needed isn't on any of the local boxes
after a server restart?
Thanks,
Paul
On Aug 31, 1:26 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> Hi,
>
> ```
> I just pushed support for a `local` gateway:http://github.com/elasticsearch/elasticsearch/issues/issue/343. The idea of
>
> ```
>
> the local gateway is to allow to perform full cluster recovery from local
> information each node stores and not require a shared storage gateway (like
> `fs` for shared file system, or `s3`, and `hdfs`).
>
> ```
> The idea of a shared gateway works really well when local node data is
>
> ```
>
> considered transient, but a full cluster recovery is still required. A
> transient local data can be when deciding to store the index in memory.
> Another benefit of using a shared storage gateway is to allow for easy
> backup of the index. If a backup process is required (on top of the high
> availability aspect of elasticsearch), it make sense to have it integrated
> into how elasticsearch works, so the recovery process will take the backups
> into account.
>
> ```
> Still, there are many cases where reusing the local data stored on each
>
> ```
>
> node when performing full recovery make sense. Very large indices for
> example, that are good with N (\>1) replicas per shard, might find the shared
> storage an overhead. Note that the shared storage model is not something
> that other nosql solution provide, and most people are ok with relying on
> local node storage and increased number of replicas. The good news is that
> elasticsearch provides it now.
>
> ```
> As a side note, shared storage model is something that takes some time
>
> ```
>
> for people to understand (I can't count the number of times I have heard
> something like: "ahh, it requires a shared storage, its not really
> distributed then..."). Its a given when talking about "in memory" data
> grids, since when bringing down the whole cluster, data was lost, and there
> should be a way to recover it. With elasticsearch, this was my first
> architecture decision for long term persistency, but I always had on the
> roadmap the mentioned support.
>
> ```
> Of course, this only make sense when using file based index storage (the
>
> ```
>
> transaction log has moved to be file based since 0.8).
>
> ```
> Enabling it is quite simple:
>
> ```
>
> gateway:
> type: "local"
>
> ```
> Usually, you would want to configure the "gateway.recover_after_nodes"
>
> ```
>
> setting to allow for more nodes to be in play when performing full cluster
> restart so the correct cluster state is elected.
>
> ```
> Last, I am considering make this setting the default one that comes with
>
> ```
>
> the elasticsearch.yml setting. What do you think?
>
> -shay.banon
---
### Author:  [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)
#### Post date: [August 31, 2010, 7:38pm UTC](https://discuss.elastic.co/t/just-pushed-local-gateway/3286/3 "2010-08-31T19:38:46Z")
Hi Kimchy
> ```
> Usually, you would want to configure the
>
> ```
>
> "gateway.recover\_after\_nodes" setting to allow for more nodes to be in
> play when performing full cluster restart so the correct cluster state
> is elected.
So what would happen if two nodes have different versions of the same
shard stored locally? I presume it'd take the more recent version. But
what it if has already restarted and used the older version?
What about situations when the cluster has split, so each node has a
version of the shard with changes that the other node doesn't have?
Is there any (efficient) way to merge these?
> ```
> Last, I am considering make this setting the default one that
>
> ```
>
> comes with the elasticsearch.yml setting. What do you think?
It would certainly make getting started simpler for the newly arrived.
Instant persistent storage without configuration.
clint
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [August 31, 2010, 8:27pm UTC](https://discuss.elastic.co/t/just-pushed-local-gateway/3286/4 "2010-08-31T20:27:59Z")
Hi,
While not all shards are found on the currently discovered nodes, the
index is in a "blocked" state and operations are not allowed on it. Note
that replicas are in this game as well.
The versioning is separated into two. The first is trying to find the
latest "meta data", which include the indices created, their settings and
mapping. This will happen once the "recover\_after\_nodes" nodes are found,
and the latest version is chosen.
Then there is the part where for each index, once all the relevant shards
are found for it, then they will be used. The "latest" shard versions will
be used and allocated.
```
If there is a split the involves more than one node (i.e. you have a 20
```
node cluster that got split in half), the way that it works now is that both
will become separated clusters, with their own versions and so on. If a full
cluster restart is done, and all 20 nodes are back into the picture, then
the one that managed to get to a higher version will win.
-shay.banon
On Tue, Aug 31, 2010 at 10:38 PM, Clinton Gormley
[clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk)wrote:
> Hi Kimchy
>
> > ```
> > Usually, you would want to configure the
> >
> > ```
> >
> > "gateway.recover\_after\_nodes" setting to allow for more nodes to be in
> > play when performing full cluster restart so the correct cluster state
> > is elected.
>
> So what would happen if two nodes have different versions of the same
> shard stored locally? I presume it'd take the more recent version. But
> what it if has already restarted and used the older version?
>
> What about situations when the cluster has split, so each node has a
> version of the shard with changes that the other node doesn't have?
>
> Is there any (efficient) way to merge these?
>
> > ```
> > Last, I am considering make this setting the default one that
> >
> > ```
> >
> > comes with the elasticsearch.yml setting. What do you think?
>
> It would certainly make getting started simpler for the newly arrived.
> Instant persistent storage without configuration.
>
> clint
---
### Author:  [@Berkay\_Mollamustafao](https://discuss.elastic.co/u/Berkay_Mollamustafao)
#### Post date: [August 31, 2010, 8:57pm UTC](https://discuss.elastic.co/t/just-pushed-local-gateway/3286/5 "2010-08-31T20:57:52Z")
+1 for making local storage the default config. Although I think most
production implementations will use shared storage, most people start with a
single instance, and shared storage is not familiar to many. Making local
storage would eliminate one more thing to learn in the beginning.
As time goes, people can see the pros/cons of shared storage themselves and
choose the right config.
Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skype
On Tue, Aug 31, 2010 at 3:26 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote:
> Hi,
>
> ```
> I just pushed support for a `local` gateway:
>
> ```
>
> [Gateway: a `local` gateway · Issue #343 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/issue/343). The idea
> of the local gateway is to allow to perform full cluster recovery from local
> information each node stores and not require a shared storage gateway (like
> `fs` for shared file system, or `s3`, and `hdfs`).
>
> ```
> The idea of a shared gateway works really well when local node data is
>
> ```
>
> considered transient, but a full cluster recovery is still required. A
> transient local data can be when deciding to store the index in memory.
> Another benefit of using a shared storage gateway is to allow for easy
> backup of the index. If a backup process is required (on top of the high
> availability aspect of elasticsearch), it make sense to have it integrated
> into how elasticsearch works, so the recovery process will take the backups
> into account.
>
> ```
> Still, there are many cases where reusing the local data stored on each
>
> ```
>
> node when performing full recovery make sense. Very large indices for
> example, that are good with N (\>1) replicas per shard, might find the shared
> storage an overhead. Note that the shared storage model is not something
> that other nosql solution provide, and most people are ok with relying on
> local node storage and increased number of replicas. The good news is that
> elasticsearch provides it now.
>
> ```
> As a side note, shared storage model is something that takes some time
>
> ```
>
> for people to understand (I can't count the number of times I have heard
> something like: "ahh, it requires a shared storage, its not really
> distributed then..."). Its a given when talking about "in memory" data
> grids, since when bringing down the whole cluster, data was lost, and there
> should be a way to recover it. With elasticsearch, this was my first
> architecture decision for long term persistency, but I always had on the
> roadmap the mentioned support.
>
> ```
> Of course, this only make sense when using file based index storage
>
> ```
>
> (the transaction log has moved to be file based since 0.8).
>
> ```
> Enabling it is quite simple:
>
> ```
>
> gateway:
> type: "local"
>
> ```
> Usually, you would want to configure the "gateway.recover_after_nodes"
>
> ```
>
> setting to allow for more nodes to be in play when performing full cluster
> restart so the correct cluster state is elected.
>
> ```
> Last, I am considering make this setting the default one that comes
>
> ```
>
> with the elasticsearch.yml setting. What do you think?
>
> -shay.banon
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [August 31, 2010, 10:50pm UTC](https://discuss.elastic.co/t/just-pushed-local-gateway/3286/6 "2010-08-31T22:50:55Z")
Yes, agreed. Just want to stress the point that "local" gateway will also
work in distributed environment, and actually, I can see it being used at
the way to support long term persistency for different type of applications
(like I noted on my first email).
-shay.banon
On Tue, Aug 31, 2010 at 11:57 PM, Berkay Mollamustafaoglu \<[mberkay@gmail.com](mailto:mberkay@gmail.com)
> wrote:
> +1 for making local storage the default config. Although I think most
> production implementations will use shared storage, most people start with a
> single instance, and shared storage is not familiar to many. Making local
> storage would eliminate one more thing to learn in the beginning.
> As time goes, people can see the pros/cons of shared storage themselves and
> choose the right config.
>
> Regards,
> Berkay Mollamustafaoglu
> mberkay on yahoo, google and skype
>
> On Tue, Aug 31, 2010 at 3:26 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote:
>
> > Hi,
> >
> > ```
> > I just pushed support for a `local` gateway:
> >
> > ```
> >
> > [Gateway: a `local` gateway · Issue #343 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/issue/343). The idea
> > of the local gateway is to allow to perform full cluster recovery from local
> > information each node stores and not require a shared storage gateway (like
> > `fs` for shared file system, or `s3`, and `hdfs`).
> >
> > ```
> > The idea of a shared gateway works really well when local node data is
> >
> > ```
> >
> > considered transient, but a full cluster recovery is still required. A
> > transient local data can be when deciding to store the index in memory.
> > Another benefit of using a shared storage gateway is to allow for easy
> > backup of the index. If a backup process is required (on top of the high
> > availability aspect of elasticsearch), it make sense to have it integrated
> > into how elasticsearch works, so the recovery process will take the backups
> > into account.
> >
> > ```
> > Still, there are many cases where reusing the local data stored on
> >
> > ```
> >
> > each node when performing full recovery make sense. Very large indices for
> > example, that are good with N (\>1) replicas per shard, might find the shared
> > storage an overhead. Note that the shared storage model is not something
> > that other nosql solution provide, and most people are ok with relying on
> > local node storage and increased number of replicas. The good news is that
> > elasticsearch provides it now.
> >
> > ```
> > As a side note, shared storage model is something that takes some time
> >
> > ```
> >
> > for people to understand (I can't count the number of times I have heard
> > something like: "ahh, it requires a shared storage, its not really
> > distributed then..."). Its a given when talking about "in memory" data
> > grids, since when bringing down the whole cluster, data was lost, and there
> > should be a way to recover it. With elasticsearch, this was my first
> > architecture decision for long term persistency, but I always had on the
> > roadmap the mentioned support.
> >
> > ```
> > Of course, this only make sense when using file based index storage
> >
> > ```
> >
> > (the transaction log has moved to be file based since 0.8).
> >
> > ```
> > Enabling it is quite simple:
> >
> > ```
> >
> > gateway:
> > type: "local"
> >
> > ```
> > Usually, you would want to configure the "gateway.recover_after_nodes"
> >
> > ```
> >
> > setting to allow for more nodes to be in play when performing full cluster
> > restart so the correct cluster state is elected.
> >
> > ```
> > Last, I am considering make this setting the default one that comes
> >
> > ```
> >
> > with the elasticsearch.yml setting. What do you think?
> >
> > -shay.banon
---
### Author:  [@Andrei](https://discuss.elastic.co/u/Andrei)
#### Post date: [September 30, 2010, 5:12am UTC](https://discuss.elastic.co/t/just-pushed-local-gateway/3286/7 "2010-09-30T05:12:08Z")
Is there any other configuration to be done for the local gateway
type, such as filesystem location, etc?
-Andrei
On Aug 31, 12:26 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> Hi,
>
> ```
> I just pushed support for a `local` gateway:http://github.com/elasticsearch/elasticsearch/issues/issue/343. The idea of
>
> ```
>
> the local gateway is to allow to perform full cluster recovery from local
> information each node stores and not require a shared storage gateway (like
> `fs` for shared file system, or `s3`, and `hdfs`).
>
> ```
> The idea of a shared gateway works really well when local node data is
>
> ```
>
> considered transient, but a full cluster recovery is still required. A
> transient local data can be when deciding to store the index in memory.
> Another benefit of using a shared storage gateway is to allow for easy
> backup of the index. If a backup process is required (on top of the high
> availability aspect of elasticsearch), it make sense to have it integrated
> into how elasticsearch works, so the recovery process will take the backups
> into account.
>
> ```
> Still, there are many cases where reusing the local data stored on each
>
> ```
>
> node when performing full recovery make sense. Very large indices for
> example, that are good with N (\>1) replicas per shard, might find the shared
> storage an overhead. Note that the shared storage model is not something
> that other nosql solution provide, and most people are ok with relying on
> local node storage and increased number of replicas. The good news is that
> elasticsearch provides it now.
>
> ```
> As a side note, shared storage model is something that takes some time
>
> ```
>
> for people to understand (I can't count the number of times I have heard
> something like: "ahh, it requires a shared storage, its not really
> distributed then..."). Its a given when talking about "in memory" data
> grids, since when bringing down the whole cluster, data was lost, and there
> should be a way to recover it. With elasticsearch, this was my first
> architecture decision for long term persistency, but I always had on the
> roadmap the mentioned support.
>
> ```
> Of course, this only make sense when using file based index storage (the
>
> ```
>
> transaction log has moved to be file based since 0.8).
>
> ```
> Enabling it is quite simple:
>
> ```
>
> gateway:
> type: "local"
>
> ```
> Usually, you would want to configure the "gateway.recover_after_nodes"
>
> ```
>
> setting to allow for more nodes to be in play when performing full cluster
> restart so the correct cluster state is elected.
>
> ```
> Last, I am considering make this setting the default one that comes with
>
> ```
>
> the elasticsearch.yml setting. What do you think?
>
> -shay.banon
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [September 30, 2010, 5:42am UTC](https://discuss.elastic.co/t/just-pushed-local-gateway/3286/8 "2010-09-30T05:42:02Z")
It defaults to the work location. If you want to change it, then you can
change the work location.
On Thu, Sep 30, 2010 at 7:12 AM, Andrei [andrei@zmievski.org](mailto:andrei@zmievski.org) wrote:
> Is there any other configuration to be done for the local gateway
> type, such as filesystem location, etc?
>
> -Andrei
>
> On Aug 31, 12:26 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
>
> > Hi,
> >
> > ```
> > I just pushed support for a `local` gateway:
> >
> > ```
>
> [Gateway: a `local` gateway · Issue #343 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/issue/343). The idea
> of
>
> > the local gateway is to allow to perform full cluster recovery from local
> > information each node stores and not require a shared storage gateway
> > (like
> > `fs` for shared file system, or `s3`, and `hdfs`).
> >
> > ```
> > The idea of a shared gateway works really well when local node data
> >
> > ```
>
> is
>
> > considered transient, but a full cluster recovery is still required. A
> > transient local data can be when deciding to store the index in memory.
> > Another benefit of using a shared storage gateway is to allow for easy
> > backup of the index. If a backup process is required (on top of the high
> > availability aspect of elasticsearch), it make sense to have it
> > integrated
> > into how elasticsearch works, so the recovery process will take the
> > backups
> > into account.
> >
> > ```
> > Still, there are many cases where reusing the local data stored on
> >
> > ```
>
> each
>
> > node when performing full recovery make sense. Very large indices for
> > example, that are good with N (\>1) replicas per shard, might find the
> > shared
> > storage an overhead. Note that the shared storage model is not something
> > that other nosql solution provide, and most people are ok with relying on
> > local node storage and increased number of replicas. The good news is
> > that
> > elasticsearch provides it now.
> >
> > ```
> > As a side note, shared storage model is something that takes some
> >
> > ```
>
> time
>
> > for people to understand (I can't count the number of times I have heard
> > something like: "ahh, it requires a shared storage, its not really
> > distributed then..."). Its a given when talking about "in memory" data
> > grids, since when bringing down the whole cluster, data was lost, and
> > there
> > should be a way to recover it. With elasticsearch, this was my first
> > architecture decision for long term persistency, but I always had on the
> > roadmap the mentioned support.
> >
> > ```
> > Of course, this only make sense when using file based index storage
> >
> > ```
>
> (the
>
> > transaction log has moved to be file based since 0.8).
> >
> > ```
> > Enabling it is quite simple:
> >
> > ```
> >
> > gateway:
> > type: "local"
> >
> > ```
> > Usually, you would want to configure the
> >
> > ```
>
> "gateway.recover\_after\_nodes"
>
> > setting to allow for more nodes to be in play when performing full
> > cluster
> > restart so the correct cluster state is elected.
> >
> > ```
> > Last, I am considering make this setting the default one that comes
> >
> > ```
>
> with
>
> > the elasticsearch.yml setting. What do you think?
> >
> > -shay.banon
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:18am UTC](https://discuss.elastic.co/t/just-pushed-local-gateway/3286/9 "2017-07-06T04:18:42Z")