# Open/close API and local gateway **URL:** \ **Category:** Elasticsearch\ **Created:** [December 9, 2010, 7:34pm UTC](https://discuss.elastic.co/t/open-close-api-and-local-gateway/3645 "2010-12-09T19:34:33Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![hartzler\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hartzler_2/32/3301_2.png) [@hartzler\_2](https://discuss.elastic.co/u/hartzler_2)\ **Post date:** [December 9, 2010, 7:34pm UTC](https://discuss.elastic.co/t/open-close-api-and-local-gateway/3645/1 "2010-12-09T19:34:33Z")
When using local gateways and then closing an index, it removes the local gateways for the index. This makes it impossible to open the index again, as the data is gone. Is this the intended behavior? The work around is to use a shared gateway (thus our implementing the cassandra gateway at Gist) as that doesn't depend on shard allocation for its location. But it has been pointed out that if the local gateways didn't get destroyed on close and reshuffled on re-open that would be just as good. Thanks! ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 9, 2010, 8:01pm UTC](https://discuss.elastic.co/t/open-close-api-and-local-gateway/3645/2 "2010-12-09T20:01:16Z")
close and open should work with local gateway, the indices should not go away. I have a test for that and it verifies that after closing all nodes and reopening them should bring back the index in the same state. Can you maybe gist the steps you do to recreate it? On Thursday, December 9, 2010 at 9:34 PM, Matt Hartzler wrote: > When using local gateways and then closing an index, it removes the local gateways for the index. This makes it impossible to open the index again, as the data is gone. Is this the intended behavior? > > The work around is to use a shared gateway (thus our implementing the cassandra gateway at Gist) as that doesn't depend on shard allocation for its location. But it has been pointed out that if the local gateways didn't get destroyed on close and reshuffled on re-open that would be just as good. > > Thanks! ---
**Author:** ![hartzler\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hartzler_2/32/3301_2.png) [@hartzler\_2](https://discuss.elastic.co/u/hartzler_2)\ **Post date:** [December 9, 2010, 10:50pm UTC](https://discuss.elastic.co/t/open-close-api-and-local-gateway/3645/3 "2010-12-09T22:50:20Z")
No I can't, the ES code works perfect. It was my own code that was doing it. Now I feel sheepish. Well good news, thats much simpler to work with! I am using a NodeAdminClient to close the indexes from a node. admin.indices().prepareClose(index).execute().actionGet(); I just want to simulate sending a "POST /index/\_close" ... however when i run that i get: [2010-12-09 16:44:41,546][DEBUG][client.node.lru] LruCache over capacity: 1 removing: 9 [2010-12-09 16:44:41,546][DEBUG][client.node.lru] Closing index: 9 ... [2010-12-09 16:44:41,546][DEBUG][cluster.service] [N0] processing [close-index [9]]: execute [2010-12-09 16:44:41,546][INFO][cluster.metadata] [N0] [9] closing index [2010-12-09 16:44:41,546][DEBUG][cluster.service] [N0] cluster state updated, version [143], source [close-index [9]] [2010-12-09 16:44:41,547][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: execute [2010-12-09 16:44:41,547][DEBUG][indices.cluster] [N0] [9][0] deleting shard [2010-12-09 16:44:41,547][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: no change in cluster\_state [2010-12-09 16:44:41,547][DEBUG][index.service] [N0] [9] deleting shard\_id [0] [2010-12-09 16:44:41,548][DEBUG][index.shard.service] [N0] [9][0] state: [STARTED]-\>[CLOSED] [2010-12-09 16:44:41,549][DEBUG][indices.memory] [N0] recalculating shard indexing buffer (reason=removed\_shard[9][0]), total is [101.9mb] with [1] shards, each shard set to [101.9mb] [2010-12-09 16:44:41,550][DEBUG][indices.cluster] [N0] [9] cleaning index (no shards allocated) [2010-12-09 16:44:41,551][DEBUG][cluster.service] [N0] processing [close-index [9]]: done applying updated cluster\_state [ instead of just [2010-12-09 16:45:00,493][DEBUG][cluster.service] [N0] processing [close-index [1]]: execute [2010-12-09 16:45:00,493][INFO][cluster.metadata] [N0] [1] closing index [2010-12-09 16:45:00,494][DEBUG][cluster.service] [N0] cluster state updated, version [144], source [close-index [1]] [2010-12-09 16:45:00,495][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: execute [2010-12-09 16:45:00,495][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: no change in cluster\_state [2010-12-09 16:45:00,495][DEBUG][cluster.service] [N0] processing [close-index [1]]: done applying updated cluster\_state [2010-12-09 16:45:00,496][DEBUG][cluster.service] [N0] processing [routing-table-updater]: execute [2010-12-09 16:45:00,497][DEBUG][cluster.service] [N0] processing [routing-table-updater]: no change in cluster\_state when i use curl (no deletes). any hints? On Thu, Dec 9, 2010 at 2:01 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > close and open should work with local gateway, the indices should not go > away. I have a test for that and it verifies that after closing all nodes > and reopening them should bring back the index in the same state. > > Can you maybe gist the steps you do to recreate it? > > On Thursday, December 9, 2010 at 9:34 PM, Matt Hartzler wrote: > > When using local gateways and then closing an index, it removes the local > gateways for the index. This makes it impossible to open the index again, > as the data is gone. Is this the intended behavior? > > The work around is to use a shared gateway (thus our implementing the > cassandra gateway at Gist) as that doesn't depend on shard allocation for > its location. But it has been pointed out that if the local gateways didn't > get destroyed on close and reshuffled on re-open that would be just as good. > > Thanks! ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 10, 2010, 3:29am UTC](https://discuss.elastic.co/t/open-close-api-and-local-gateway/3645/4 "2010-12-10T03:29:44Z")
I did not understand your question. Are things working or not? On Friday, December 10, 2010 at 12:50 AM, Matt Hartzler wrote: > No I can't, the ES code works perfect. It was my own code that was doing it. Now I feel sheepish. Well good news, thats much simpler to work with! > > I am using a NodeAdminClient to close the indexes from a node. > > admin.indices().prepareClose(index).execute().actionGet(); > > I just want to simulate sending a "POST /index/\_close" ... > > however when i run that i get: > > [2010-12-09 16:44:41,546][DEBUG][client.node.lru] LruCache over capacity: 1 removing: 9 > [2010-12-09 16:44:41,546][DEBUG][client.node.lru] Closing index: 9 ... > [2010-12-09 16:44:41,546][DEBUG][cluster.service] [N0] processing [close-index [9]]: execute > [2010-12-09 16:44:41,546][INFO][cluster.metadata] [N0] [9] closing index > [2010-12-09 16:44:41,546][DEBUG][cluster.service] [N0] cluster state updated, version [143], source [close-index [9]] > [2010-12-09 16:44:41,547][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: execute > [2010-12-09 16:44:41,547][DEBUG][indices.cluster] [N0] [9][0] deleting shard > [2010-12-09 16:44:41,547][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: no change in cluster\_state > [2010-12-09 16:44:41,547][DEBUG][index.service] [N0] [9] deleting shard\_id [0] > [2010-12-09 16:44:41,548][DEBUG][index.shard.service] [N0] [9][0] state: [STARTED]-\>[CLOSED] > [2010-12-09 16:44:41,549][DEBUG][indices.memory] [N0] recalculating shard indexing buffer (reason=removed\_shard[9][0]), total is [101.9mb] with [1] shards, each shard set to [101.9mb] > [2010-12-09 16:44:41,550][DEBUG][indices.cluster] [N0] [9] cleaning index (no shards allocated) > [2010-12-09 16:44:41,551][DEBUG][cluster.service] [N0] processing [close-index [9]]: done applying updated cluster\_state > [ > > instead of just > > [2010-12-09 16:45:00,493][DEBUG][cluster.service] [N0] processing [close-index [1]]: execute > [2010-12-09 16:45:00,493][INFO][cluster.metadata] [N0] [1] closing index > [2010-12-09 16:45:00,494][DEBUG][cluster.service] [N0] cluster state updated, version [144], source [close-index [1]] > [2010-12-09 16:45:00,495][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: execute > [2010-12-09 16:45:00,495][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: no change in cluster\_state > [2010-12-09 16:45:00,495][DEBUG][cluster.service] [N0] processing [close-index [1]]: done applying updated cluster\_state > [2010-12-09 16:45:00,496][DEBUG][cluster.service] [N0] processing [routing-table-updater]: execute > [2010-12-09 16:45:00,497][DEBUG][cluster.service] [N0] processing [routing-table-updater]: no change in cluster\_state > > when i use curl (no deletes). any hints? > > On Thu, Dec 9, 2010 at 2:01 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > close and open should work with local gateway, the indices should not go away. I have a test for that and it verifies that after closing all nodes and reopening them should bring back the index in the same state. > > > > Can you maybe gist the steps you do to recreate it? > > > > On Thursday, December 9, 2010 at 9:34 PM, Matt Hartzler wrote: > > > > > When using local gateways and then closing an index, it removes the local gateways for the index. This makes it impossible to open the index again, as the data is gone. Is this the intended behavior? > > > > > > The work around is to use a shared gateway (thus our implementing the cassandra gateway at Gist) as that doesn't depend on shard allocation for its location. But it has been pointed out that if the local gateways didn't get destroyed on close and reshuffled on re-open that would be just as good. > > > > > > Thanks! ---
**Author:** ![hartzler\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hartzler_2/32/3301_2.png) [@hartzler\_2](https://discuss.elastic.co/u/hartzler_2)\ **Post date:** [December 10, 2010, 5:05pm UTC](https://discuss.elastic.co/t/open-close-api-and-local-gateway/3645/5 "2010-12-10T17:05:08Z")
It is not working. Here is the gist: [demonstration of open/close API nuking local gateway index · GitHub](https://gist.github.com/736467) Sorry for the confusion, I was just having trouble tracking down what was going wrong. Looks like once you create a new index, the closed index gets deleted. On Thu, Dec 9, 2010 at 9:29 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > I did not understand your question. Are things working or not? > > On Friday, December 10, 2010 at 12:50 AM, Matt Hartzler wrote: > > No I can't, the ES code works perfect. It was my own code that was doing > it. Now I feel sheepish. Well good news, thats much simpler to work with! > > I am using a NodeAdminClient to close the indexes from a node. > > admin.indices().prepareClose(index).execute().actionGet(); > > I just want to simulate sending a "POST /index/\_close" ... > > however when i run that i get: > > [2010-12-09 16:44:41,546][DEBUG][client.node.lru] LruCache over > capacity: 1 removing: 9 > [2010-12-09 16:44:41,546][DEBUG][client.node.lru] Closing index: > 9 ... > [2010-12-09 16:44:41,546][DEBUG][cluster.service] [N0] processing > [close-index [9]]: execute > [2010-12-09 16:44:41,546][INFO][cluster.metadata] [N0] [9] > closing index > [2010-12-09 16:44:41,546][DEBUG][cluster.service] [N0] cluster > state updated, version [143], source [close-index [9]] > [2010-12-09 16:44:41,547][DEBUG][river.cluster] [N0] processing > [reroute\_rivers\_node\_changed]: execute > [2010-12-09 16:44:41,547][DEBUG][indices.cluster] [N0] [9][0] > deleting shard > [2010-12-09 16:44:41,547][DEBUG][river.cluster] [N0] processing > [reroute\_rivers\_node\_changed]: no change in cluster\_state > [2010-12-09 16:44:41,547][DEBUG][index.service] [N0] [9] > deleting shard\_id [0] > [2010-12-09 16:44:41,548][DEBUG][index.shard.service] [N0] [9][0] > state: [STARTED]-\>[CLOSED] > [2010-12-09 16:44:41,549][DEBUG][indices.memory] [N0] > recalculating shard indexing buffer (reason=removed\_shard[9][0]), total is > [101.9mb] with [1] shards, each shard set to [101.9mb] > [2010-12-09 16:44:41,550][DEBUG][indices.cluster] [N0] [9] > cleaning index (no shards allocated) > [2010-12-09 16:44:41,551][DEBUG][cluster.service] [N0] processing > [close-index [9]]: done applying updated cluster\_state > [ > > instead of just > > [2010-12-09 16:45:00,493][DEBUG][cluster.service] [N0] processing > [close-index [1]]: execute > [2010-12-09 16:45:00,493][INFO][cluster.metadata] [N0] [1] > closing index > [2010-12-09 16:45:00,494][DEBUG][cluster.service] [N0] cluster > state updated, version [144], source [close-index [1]] > [2010-12-09 16:45:00,495][DEBUG][river.cluster] [N0] processing > [reroute\_rivers\_node\_changed]: execute > [2010-12-09 16:45:00,495][DEBUG][river.cluster] [N0] processing > [reroute\_rivers\_node\_changed]: no change in cluster\_state > [2010-12-09 16:45:00,495][DEBUG][cluster.service] [N0] processing > [close-index [1]]: done applying updated cluster\_state > [2010-12-09 16:45:00,496][DEBUG][cluster.service] [N0] processing > [routing-table-updater]: execute > [2010-12-09 16:45:00,497][DEBUG][cluster.service] [N0] processing > [routing-table-updater]: no change in cluster\_state > > when i use curl (no deletes). any hints? > > On Thu, Dec 9, 2010 at 2:01 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > > close and open should work with local gateway, the indices should not go > away. I have a test for that and it verifies that after closing all nodes > and reopening them should bring back the index in the same state. > > Can you maybe gist the steps you do to recreate it? > > On Thursday, December 9, 2010 at 9:34 PM, Matt Hartzler wrote: > > When using local gateways and then closing an index, it removes the local > gateways for the index. This makes it impossible to open the index again, > as the data is gone. Is this the intended behavior? > > The work around is to use a shared gateway (thus our implementing the > cassandra gateway at Gist) as that doesn't depend on shard allocation for > its location. But it has been pointed out that if the local gateways didn't > get destroyed on close and reshuffled on re-open that would be just as good. > > Thanks! ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 10, 2010, 8:20pm UTC](https://discuss.elastic.co/t/open-close-api-and-local-gateway/3645/6 "2010-12-10T20:20:49Z")
Hey, Found a bug in the close API, which might cause the data for that index to be wiped... . Fixed it in master, great for catching this!. [Close API might cause index data to be wiped · Issue #560 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/issue/560/). -shay.banon On Friday, December 10, 2010 at 7:05 PM, Matt Hartzler wrote: > It is not working. Here is the gist: [demonstration of open/close API nuking local gateway index · GitHub](https://gist.github.com/736467) > > Sorry for the confusion, I was just having trouble tracking down what was going wrong. Looks like once you create a new index, the closed index gets deleted. > > On Thu, Dec 9, 2010 at 9:29 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > I did not understand your question. Are things working or not? > > > > On Friday, December 10, 2010 at 12:50 AM, Matt Hartzler wrote: > > > > > No I can't, the ES code works perfect. It was my own code that was doing it. Now I feel sheepish. Well good news, thats much simpler to work with! > > > > > > I am using a NodeAdminClient to close the indexes from a node. > > > > > > admin.indices().prepareClose(index).execute().actionGet(); > > > > > > I just want to simulate sending a "POST /index/\_close" ... > > > > > > however when i run that i get: > > > > > > [2010-12-09 16:44:41,546][DEBUG][client.node.lru] LruCache over capacity: 1 removing: 9 > > > [2010-12-09 16:44:41,546][DEBUG][client.node.lru] Closing index: 9 ... > > > [2010-12-09 16:44:41,546][DEBUG][cluster.service] [N0] processing [close-index [9]]: execute > > > [2010-12-09 16:44:41,546][INFO][cluster.metadata] [N0] [9] closing index > > > [2010-12-09 16:44:41,546][DEBUG][cluster.service] [N0] cluster state updated, version [143], source [close-index [9]] > > > [2010-12-09 16:44:41,547][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: execute > > > [2010-12-09 16:44:41,547][DEBUG][indices.cluster] [N0] [9][0] deleting shard > > > [2010-12-09 16:44:41,547][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: no change in cluster\_state > > > [2010-12-09 16:44:41,547][DEBUG][index.service] [N0] [9] deleting shard\_id [0] > > > [2010-12-09 16:44:41,548][DEBUG][index.shard.service] [N0] [9][0] state: [STARTED]-\>[CLOSED] > > > [2010-12-09 16:44:41,549][DEBUG][indices.memory] [N0] recalculating shard indexing buffer (reason=removed\_shard[9][0]), total is [101.9mb] with [1] shards, each shard set to [101.9mb] > > > [2010-12-09 16:44:41,550][DEBUG][indices.cluster] [N0] [9] cleaning index (no shards allocated) > > > [2010-12-09 16:44:41,551][DEBUG][cluster.service] [N0] processing [close-index [9]]: done applying updated cluster\_state > > > [ > > > > > > instead of just > > > > > > [2010-12-09 16:45:00,493][DEBUG][cluster.service] [N0] processing [close-index [1]]: execute > > > [2010-12-09 16:45:00,493][INFO][cluster.metadata] [N0] [1] closing index > > > [2010-12-09 16:45:00,494][DEBUG][cluster.service] [N0] cluster state updated, version [144], source [close-index [1]] > > > [2010-12-09 16:45:00,495][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: execute > > > [2010-12-09 16:45:00,495][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: no change in cluster\_state > > > [2010-12-09 16:45:00,495][DEBUG][cluster.service] [N0] processing [close-index [1]]: done applying updated cluster\_state > > > [2010-12-09 16:45:00,496][DEBUG][cluster.service] [N0] processing [routing-table-updater]: execute > > > [2010-12-09 16:45:00,497][DEBUG][cluster.service] [N0] processing [routing-table-updater]: no change in cluster\_state > > > > > > when i use curl (no deletes). any hints? > > > > > > On Thu, Dec 9, 2010 at 2:01 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > > > > > > > close and open should work with local gateway, the indices should not go away. I have a test for that and it verifies that after closing all nodes and reopening them should bring back the index in the same state. > > > > > > > > Can you maybe gist the steps you do to recreate it? > > > > > > > > On Thursday, December 9, 2010 at 9:34 PM, Matt Hartzler wrote: > > > > > > > > > When using local gateways and then closing an index, it removes the local gateways for the index. This makes it impossible to open the index again, as the data is gone. Is this the intended behavior? > > > > > > > > > > The work around is to use a shared gateway (thus our implementing the cassandra gateway at Gist) as that doesn't depend on shard allocation for its location. But it has been pointed out that if the local gateways didn't get destroyed on close and reshuffled on re-open that would be just as good. > > > > > > > > > > Thanks! ---
**Author:** ![tommay](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tommay/32/3295_2.png) [@tommay](https://discuss.elastic.co/u/tommay)\ **Post date:** [December 15, 2010, 12:52am UTC](https://discuss.elastic.co/t/open-close-api-and-local-gateway/3645/7 "2010-12-15T00:52:41Z")
Even with your fix, Matt's repro still fails the same way if index.store.type is set to memory. I'm pretty sure that's supposed to work: the gateway has a persistent copy of the index and the index should be restored from the gateway when it's opened. It works fine if the gateway is fs, but not local. I.e., it still fails with this elasticsearch.yml: network: tcp: keep\_alive: true host: 127.0.0.1 cluster: name: development gateway: type: local index: store: type: memory number\_of\_shards: 1 number\_of\_replicas: 0 robin: refreshInterval: -1 Tom On Fri, Dec 10, 2010 at 12:20 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > Hey, > > Found a bug in the close API, which might cause the data for that index > to be wiped... . Fixed it in master, great for catching this!. > [Close API might cause index data to be wiped · Issue #560 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/issue/560/). > > -shay.banon > > On Friday, December 10, 2010 at 7:05 PM, Matt Hartzler wrote: > > It is not working. Here is the gist: [demonstration of open/close API nuking local gateway index · GitHub](https://gist.github.com/736467) > > Sorry for the confusion, I was just having trouble tracking down what was > going wrong. Looks like once you create a new index, the closed index gets > deleted. > > On Thu, Dec 9, 2010 at 9:29 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > > I did not understand your question. Are things working or not? > > On Friday, December 10, 2010 at 12:50 AM, Matt Hartzler wrote: > > No I can't, the ES code works perfect. It was my own code that was doing > it. Now I feel sheepish. Well good news, thats much simpler to work with! > > I am using a NodeAdminClient to close the indexes from a node. > > admin.indices().prepareClose(index).execute().actionGet(); > > I just want to simulate sending a "POST /index/\_close" ... > > however when i run that i get: > > [2010-12-09 16:44:41,546][DEBUG][client.node.lru] LruCache over > capacity: 1 removing: 9 > [2010-12-09 16:44:41,546][DEBUG][client.node.lru] Closing index: > 9 ... > [2010-12-09 16:44:41,546][DEBUG][cluster.service] [N0] processing > [close-index [9]]: execute > [2010-12-09 16:44:41,546][INFO][cluster.metadata] [N0] [9] > closing index > [2010-12-09 16:44:41,546][DEBUG][cluster.service] [N0] cluster > state updated, version [143], source [close-index [9]] > [2010-12-09 16:44:41,547][DEBUG][river.cluster] [N0] processing > [reroute\_rivers\_node\_changed]: execute > [2010-12-09 16:44:41,547][DEBUG][indices.cluster] [N0] [9][0] > deleting shard > [2010-12-09 16:44:41,547][DEBUG][river.cluster] [N0] processing > [reroute\_rivers\_node\_changed]: no change in cluster\_state > [2010-12-09 16:44:41,547][DEBUG][index.service] [N0] [9] > deleting shard\_id [0] > [2010-12-09 16:44:41,548][DEBUG][index.shard.service] [N0] [9][0] > state: [STARTED]-\>[CLOSED] > [2010-12-09 16:44:41,549][DEBUG][indices.memory] [N0] > recalculating shard indexing buffer (reason=removed\_shard[9][0]), total is > [101.9mb] with [1] shards, each shard set to [101.9mb] > [2010-12-09 16:44:41,550][DEBUG][indices.cluster] [N0] [9] > cleaning index (no shards allocated) > [2010-12-09 16:44:41,551][DEBUG][cluster.service] [N0] processing > [close-index [9]]: done applying updated cluster\_state > [ > > instead of just > > [2010-12-09 16:45:00,493][DEBUG][cluster.service] [N0] processing > [close-index [1]]: execute > [2010-12-09 16:45:00,493][INFO][cluster.metadata] [N0] [1] > closing index > [2010-12-09 16:45:00,494][DEBUG][cluster.service] [N0] cluster > state updated, version [144], source [close-index [1]] > [2010-12-09 16:45:00,495][DEBUG][river.cluster] [N0] processing > [reroute\_rivers\_node\_changed]: execute > [2010-12-09 16:45:00,495][DEBUG][river.cluster] [N0] processing > [reroute\_rivers\_node\_changed]: no change in cluster\_state > [2010-12-09 16:45:00,495][DEBUG][cluster.service] [N0] processing > [close-index [1]]: done applying updated cluster\_state > [2010-12-09 16:45:00,496][DEBUG][cluster.service] [N0] processing > [routing-table-updater]: execute > [2010-12-09 16:45:00,497][DEBUG][cluster.service] [N0] processing > [routing-table-updater]: no change in cluster\_state > > when i use curl (no deletes). any hints? > > On Thu, Dec 9, 2010 at 2:01 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > > close and open should work with local gateway, the indices should not go > away. I have a test for that and it verifies that after closing all nodes > and reopening them should bring back the index in the same state. > > Can you maybe gist the steps you do to recreate it? > > On Thursday, December 9, 2010 at 9:34 PM, Matt Hartzler wrote: > > When using local gateways and then closing an index, it removes the local > gateways for the index. This makes it impossible to open the index again, > as the data is gone. Is this the intended behavior? > > The work around is to use a shared gateway (thus our implementing the > cassandra gateway at Gist) as that doesn't depend on shard allocation for > its location. But it has been pointed out that if the local gateways didn't > get destroyed on close and reshuffled on re-open that would be just as good. > > Thanks! ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 15, 2010, 5:42pm UTC](https://discuss.elastic.co/t/open-close-api-and-local-gateway/3645/8 "2010-12-15T17:42:34Z")
If index.store.type is set to memory, then you can't restore the state of it after restart when using local gateway. On Wednesday, December 15, 2010 at 2:52 AM, Tom May wrote: Even with your fix, Matt's repro still fails the same way if index.store.type is set to memory. I'm pretty sure that's supposed to work: the gateway has a persistent copy of the index and the index should be restored from the gateway when it's opened. It works fine if the gateway is fs, but not local. I.e., it still fails with this elasticsearch.yml: network: tcp: keep\_alive: true host: 127.0.0.1 cluster: name: development gateway: type: local index: store: type: memory number\_of\_shards: 1 number\_of\_replicas: 0 robin: refreshInterval: -1 Tom On Fri, Dec 10, 2010 at 12:20 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: Hey, Found a bug in the close API, which might cause the data for that index to be wiped... . Fixed it in master, great for catching this!. [https://github.com/elasticsearch/elasticsearch/issues/issue/560/](https://github.com/elasticsearch/elasticsearch/issues/issue/560/). -shay.banon On Friday, December 10, 2010 at 7:05 PM, Matt Hartzler wrote: It is not working. Here is the gist: [https://gist.github.com/736467](https://gist.github.com/736467) Sorry for the confusion, I was just having trouble tracking down what was going wrong. Looks like once you create a new index, the closed index gets deleted. On Thu, Dec 9, 2010 at 9:29 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: I did not understand your question. Are things working or not? On Friday, December 10, 2010 at 12:50 AM, Matt Hartzler wrote: No I can't, the ES code works perfect. It was my own code that was doing it. Now I feel sheepish. Well good news, thats much simpler to work with! I am using a NodeAdminClient to close the indexes from a node. admin.indices().prepareClose(index).execute().actionGet(); I just want to simulate sending a "POST /index/\_close" ... however when i run that i get: [2010-12-09 16:44:41,546][DEBUG][client.node.lru] LruCache over capacity: 1 removing: 9 [2010-12-09 16:44:41,546][DEBUG][client.node.lru] Closing index: 9 ... [2010-12-09 16:44:41,546][DEBUG][cluster.service] [N0] processing [close-index [9]]: execute [2010-12-09 16:44:41,546][INFO][cluster.metadata] [N0] [9] closing index [2010-12-09 16:44:41,546][DEBUG][cluster.service] [N0] cluster state updated, version [143], source [close-index [9]] [2010-12-09 16:44:41,547][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: execute [2010-12-09 16:44:41,547][DEBUG][indices.cluster] [N0] [9][0] deleting shard [2010-12-09 16:44:41,547][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: no change in cluster\_state [2010-12-09 16:44:41,547][DEBUG][index.service] [N0] [9] deleting shard\_id [0] [2010-12-09 16:44:41,548][DEBUG][index.shard.service] [N0] [9][0] state: [STARTED]-\>[CLOSED] [2010-12-09 16:44:41,549][DEBUG][indices.memory] [N0] recalculating shard indexing buffer (reason=removed\_shard[9][0]), total is [101.9mb] with [1] shards, each shard set to [101.9mb] [2010-12-09 16:44:41,550][DEBUG][indices.cluster] [N0] [9] cleaning index (no shards allocated) [2010-12-09 16:44:41,551][DEBUG][cluster.service] [N0] processing [close-index [9]]: done applying updated cluster\_state [ instead of just [2010-12-09 16:45:00,493][DEBUG][cluster.service] [N0] processing [close-index [1]]: execute [2010-12-09 16:45:00,493][INFO][cluster.metadata] [N0] [1] closing index [2010-12-09 16:45:00,494][DEBUG][cluster.service] [N0] cluster state updated, version [144], source [close-index [1]] [2010-12-09 16:45:00,495][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: execute [2010-12-09 16:45:00,495][DEBUG][river.cluster] [N0] processing [reroute\_rivers\_node\_changed]: no change in cluster\_state [2010-12-09 16:45:00,495][DEBUG][cluster.service] [N0] processing [close-index [1]]: done applying updated cluster\_state [2010-12-09 16:45:00,496][DEBUG][cluster.service] [N0] processing [routing-table-updater]: execute [2010-12-09 16:45:00,497][DEBUG][cluster.service] [N0] processing [routing-table-updater]: no change in cluster\_state when i use curl (no deletes). any hints? On Thu, Dec 9, 2010 at 2:01 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: close and open should work with local gateway, the indices should not go away. I have a test for that and it verifies that after closing all nodes and reopening them should bring back the index in the same state. Can you maybe gist the steps you do to recreate it? On Thursday, December 9, 2010 at 9:34 PM, Matt Hartzler wrote: When using local gateways and then closing an index, it removes the local gateways for the index. This makes it impossible to open the index again, as the data is gone. Is this the intended behavior? The work around is to use a shared gateway (thus our implementing the cassandra gateway at Gist) as that doesn't depend on shard allocation for its location. But it has been pointed out that if the local gateways didn't get destroyed on close and reshuffled on re-open that would be just as good. Thanks! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:15am UTC](https://discuss.elastic.co/t/open-close-api-and-local-gateway/3645/9 "2017-07-06T04:15:17Z")