Need to sync Index list/status in a cluster

Elastic Stack Elasticsearch
1

Hi,

We're getting the current index list with this code:

Map<String,IndexStatus> indices= client.admin().indices().status(new
IndicesStatusRequest()).actionGet().getIndices();

But if someone removes an index then the code above return the old
list on a second node but it returns the updated list in the master
node.

After a couple of minutes they get sync. But we need a way to force
that synchronization.

We've try with:

client.admin().indices().clearCache(new
ClearIndicesCacheRequest()).actionGet();
client.admin().indices().refresh(new
RefreshRequest().waitForOperations(true)).actionGet();
client.admin().indices().flush(new
FlushRequest().full(true).force(true)).actionGet();

in both nodes but still the second node can't get the updated list
inmediatly.

Any idea?

2

Couple of minutes sounds like a lot, when you delete an index, the fact that its deleted gets applied on the elected master node, and then broadcasted to the cluster nodes. By default, the delete operation will wait for an ack from all the other nodes for 10 seconds and return (that the ack flag on the delete response). Clearing caches has nothing to do with it.

What are information do you get from the index status?

On Tuesday, January 24, 2012 at 10:13 PM, Jorge Urdaneta wrote:

Hi,

We're getting the current index list with this code:

Map<String,IndexStatus> indices= client.admin().indices().status(new
IndicesStatusRequest()).actionGet().getIndices();

But if someone removes an index then the code above return the old
list on a second node but it returns the updated list in the master
node.

After a couple of minutes they get sync. But we need a way to force
that synchronization.

We've try with:

client.admin().indices().clearCache(new
ClearIndicesCacheRequest()).actionGet();
client.admin().indices().refresh(new
RefreshRequest().waitForOperations(true)).actionGet();
client.admin().indices().flush(new
FlushRequest().full(true).force(true)).actionGet();

in both nodes but still the second node can't get the updated list
inmediatly.

Any idea?

3

I get all indexes including those I just deleted. It is a lot more
than 10 seconds I see it takes to sync all nodes. Maybe 2 minutes is a
good aproximate here. But if I call client.admin.indices.exists() on
those deleted indexes it tells false. So, that is the workarround. I
have to validate the index list returned by the index status using
exists calls on each one.

On Jan 25, 12:29 pm, Shay Banon kim...@gmail.com wrote:

Couple of minutes sounds like a lot, when you delete an index, the fact that its deleted gets applied on the elected master node, and then broadcasted to the cluster nodes. By default, the delete operation will wait for an ack from all the other nodes for 10 seconds and return (that the ack flag on the delete response). Clearing caches has nothing to do with it.

What are information do you get from the index status?

On Tuesday, January 24, 2012 at 10:13 PM, Jorge Urdaneta wrote:

Hi,

We're getting the current index list with this code:

Map<String,IndexStatus> indices= client.admin().indices().status(new
IndicesStatusRequest()).actionGet().getIndices();

But if someone removes an index then the code above return the old
list on a second node but it returns the updated list in the master
node.

After a couple of minutes they get sync. But we need a way to force
that synchronization.

We've try with:

client.admin().indices().clearCache(new
ClearIndicesCacheRequest()).actionGet();
client.admin().indices().refresh(new
RefreshRequest().waitForOperations(true)).actionGet();
client.admin().indices().flush(new
FlushRequest().full(true).force(true)).actionGet();

in both nodes but still the second node can't get the updated list
inmediatly.

Any idea?

4

index exists will go to the master node, while indices status will go to all the relevant nodes the indices exists on. You can also use cluster state (it really depends what data you need). It is still very strange that it takes this long, do you do any heavy index / cluster metadata changes?

On Thursday, January 26, 2012 at 5:59 AM, Jorge Urdaneta wrote:

I get all indexes including those I just deleted. It is a lot more
than 10 seconds I see it takes to sync all nodes. Maybe 2 minutes is a
good aproximate here. But if I call client.admin.indices.exists() on
those deleted indexes it tells false. So, that is the workarround. I
have to validate the index list returned by the index status using
exists calls on each one.

On Jan 25, 12:29 pm, Shay Banon <kim...@gmail.com (http://gmail.com)> wrote:

Couple of minutes sounds like a lot, when you delete an index, the fact that its deleted gets applied on the elected master node, and then broadcasted to the cluster nodes. By default, the delete operation will wait for an ack from all the other nodes for 10 seconds and return (that the ack flag on the delete response). Clearing caches has nothing to do with it.

What are information do you get from the index status?

On Tuesday, January 24, 2012 at 10:13 PM, Jorge Urdaneta wrote:

Hi,

We're getting the current index list with this code:

Map<String,IndexStatus> indices= client.admin().indices().status(new
IndicesStatusRequest()).actionGet().getIndices();

But if someone removes an index then the code above return the old
list on a second node but it returns the updated list in the master
node.

After a couple of minutes they get sync. But we need a way to force
that synchronization.

We've try with:

client.admin().indices().clearCache(new
ClearIndicesCacheRequest()).actionGet();
client.admin().indices().refresh(new
RefreshRequest().waitForOperations(true)).actionGet();
client.admin().indices().flush(new
FlushRequest().full(true).force(true)).actionGet();

in both nodes but still the second node can't get the updated list
inmediatly.

Any idea?

5

Yeah

It happends when we reconstruct our indexes and put there all our
documents in order to have a clean index. As it uses dynamic mapping
there is a lot of metadata changes.

On Jan 26, 5:55 am, Shay Banon kim...@gmail.com wrote:

index exists will go to the master node, while indices status will go to all the relevant nodes the indices exists on. You can also use cluster state (it really depends what data you need). It is still very strange that it takes this long, do you do any heavy index / cluster metadata changes?

On Thursday, January 26, 2012 at 5:59 AM, Jorge Urdaneta wrote:

I get all indexes including those I just deleted. It is a lot more
than 10 seconds I see it takes to sync all nodes. Maybe 2 minutes is a
good aproximate here. But if I call client.admin.indices.exists() on
those deleted indexes it tells false. So, that is the workarround. I
have to validate the index list returned by the index status using
exists calls on each one.

On Jan 25, 12:29 pm, Shay Banon <kim...@gmail.com (http://gmail.com)> wrote:

Couple of minutes sounds like a lot, when you delete an index, the fact that its deleted gets applied on the elected master node, and then broadcasted to the cluster nodes. By default, the delete operation will wait for an ack from all the other nodes for 10 seconds and return (that the ack flag on the delete response). Clearing caches has nothing to do with it.

What are information do you get from the index status?

On Tuesday, January 24, 2012 at 10:13 PM, Jorge Urdaneta wrote:

Hi,

We're getting the current index list with this code:

Map<String,IndexStatus> indices= client.admin().indices().status(new
IndicesStatusRequest()).actionGet().getIndices();

But if someone removes an index then the code above return the old
list on a second node but it returns the updated list in the master
node.

After a couple of minutes they get sync. But we need a way to force
that synchronization.

We've try with:

client.admin().indices().clearCache(new
ClearIndicesCacheRequest()).actionGet();
client.admin().indices().refresh(new
RefreshRequest().waitForOperations(true)).actionGet();
client.admin().indices().flush(new
FlushRequest().full(true).force(true)).actionGet();

in both nodes but still the second node can't get the updated list
inmediatly.

Any idea?