ClusterBlockException during integration test

Alois_Cochard · March 18, 2011, 9:28am

Hey all !

I've some simple integration test (add an entry then get it, or simple
stuff like that) that start a node, from time to time ([pseudo]-
randomly) I've got the following exception:

org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [1/
not recovered from gateway];
at
org.elasticsearch.cluster.block.ClusterBlocks.indexBlockedRaiseException(ClusterBlocks.java:
112)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:
65)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.checkBlock(TransportCreateIndexAction.java:
42)
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction
$1.run(TransportMasterNodeOperationAction.java:91)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

Does anyone experimented the same issue ? any idea why this happen ?

Thanks a lot,

Alois Cochard

http://twitter.com/aloiscochard

Alois_Cochard · March 18, 2011, 9:30am

Didn't made a search before posting (shame on me) so I found that:
http://groups.google.com/a/elasticsearch.com/group/users/browse_thread/thread/6d37daf47d667874/8a18be1d4760b18a?lnk=gst&q=ClusterBlockException#8a18be1d4760b18a

But... that's look a bit tricky, any advice Shay ?

On Mar 18, 10:28 am, Alois Cochard alois.coch...@gmail.com wrote:

Hey all !

I've some simple integration test (add an entry then get it, or simple
stuff like that) that start a node, from time to time ([pseudo]-
randomly) I've got the following exception:

org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [1/
not recovered from gateway];
at
org.elasticsearch.cluster.block.ClusterBlocks.indexBlockedRaiseException(Cl usterBlocks.java:
112)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.ch eckBlock(TransportCreateIndexAction.java:
65)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.ch eckBlock(TransportCreateIndexAction.java:
42)
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction
$1.run(TransportMasterNodeOperationAction.java:91)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

Does anyone experimented the same issue ? any idea why this happen ?

Thanks a lot,

Alois Cochardhttp://aloiscochard.blogspot.comhttp://twitter.com/aloiscochard

kimchy · March 18, 2011, 9:37pm

Which version are you using? the create index API will now automatically retry if its blocked. In any case, you can also wait for yellow cluster state.
On Friday, March 18, 2011 at 11:30 AM, Alois Cochard wrote:

Didn't made a search before posting (shame on me) so I found that:
http://groups.google.com/a/elasticsearch.com/group/users/browse_thread/thread/6d37daf47d667874/8a18be1d4760b18a?lnk=gst&q=ClusterBlockException#8a18be1d4760b18a

But... that's look a bit tricky, any advice Shay ?

On Mar 18, 10:28 am, Alois Cochard alois.coch...@gmail.com wrote:

Hey all !

I've some simple integration test (add an entry then get it, or simple
stuff like that) that start a node, from time to time ([pseudo]-
randomly) I've got the following exception:

org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [1/
not recovered from gateway];
at
org.elasticsearch.cluster.block.ClusterBlocks.indexBlockedRaiseException(Cl usterBlocks.java:
112)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.ch eckBlock(TransportCreateIndexAction.java:
65)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.ch eckBlock(TransportCreateIndexAction.java:
42)
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction
$1.run(TransportMasterNodeOperationAction.java:91)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

Does anyone experimented the same issue ? any idea why this happen ?

Thanks a lot,

Alois Cochardhttp://aloiscochard.blogspot.comhttp://twitter.com/aloiscochard

Alois_Cochard · March 21, 2011, 1:16pm

Ok,

Was using version 0.14.0, I'll update ASAP and see if problem still
occure.

Thanks a lot !

On Mar 18, 10:37 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Which version are you using? the create index API will now automatically retry if its blocked. In any case, you can also wait for yellow cluster state.

On Friday, March 18, 2011 at 11:30 AM, Alois Cochard wrote:

Didn't made a search before posting (shame on me) so I found that:
http://groups.google.com/a/elasticsearch.com/group/users/browse_threa...

But... that's look a bit tricky, any advice Shay ?

On Mar 18, 10:28 am, Alois Cochard alois.coch...@gmail.com wrote:

Hey all !

I've some simple integration test (add an entry then get it, or simple
stuff like that) that start a node, from time to time ([pseudo]-
randomly) I've got the following exception:

org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [1/
not recovered from gateway];
at
org.elasticsearch.cluster.block.ClusterBlocks.indexBlockedRaiseException(Cl usterBlocks.java:
112)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.ch eckBlock(TransportCreateIndexAction.java:
65)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.ch eckBlock(TransportCreateIndexAction.java:
42)
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction
$1.run(TransportMasterNodeOperationAction.java:91)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

Does anyone experimented the same issue ? any idea why this happen ?

Thanks a lot,

Alois Cochardhttp://aloiscochard.blogspot.comhttp://twitter.com/aloiscochard

Alois_Cochard · March 25, 2011, 9:24am

Just for information was still falling pseudo-randomly with last
version (0.15.2), don't know how many retry create index does...

But after adding the cluster health everything rock...

Cheers,

On Mar 21, 2:16 pm, Alois Cochard alois.coch...@gmail.com wrote:

Ok,

Was using version 0.14.0, I'll update ASAP and see if problem still
occure.

Thanks a lot !

On Mar 18, 10:37 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Which version are you using? the create index API will now automatically retry if its blocked. In any case, you can also wait for yellow cluster state.

On Friday, March 18, 2011 at 11:30 AM, Alois Cochard wrote:

Didn't made a search before posting (shame on me) so I found that:
http://groups.google.com/a/elasticsearch.com/group/users/browse_threa...

But... that's look a bit tricky, any advice Shay ?

On Mar 18, 10:28 am, Alois Cochard alois.coch...@gmail.com wrote:

Hey all !

I've some simple integration test (add an entry then get it, or simple
stuff like that) that start a node, from time to time ([pseudo]-
randomly) I've got the following exception:

org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [1/
not recovered from gateway];
at
org.elasticsearch.cluster.block.ClusterBlocks.indexBlockedRaiseException(Cl usterBlocks.java:
112)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.ch eckBlock(TransportCreateIndexAction.java:
65)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.ch eckBlock(TransportCreateIndexAction.java:
42)
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction
$1.run(TransportMasterNodeOperationAction.java:91)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

Does anyone experimented the same issue ? any idea why this happen ?

Thanks a lot,

Alois Cochardhttp://aloiscochard.blogspot.comhttp://twitter.com/aloiscochard

kimchy · March 25, 2011, 7:05pm

If you can recreate a simple testcase, I can give it a spin.
On Friday, March 25, 2011 at 11:24 AM, Alois Cochard wrote:

Just for information was still falling pseudo-randomly with last
version (0.15.2), don't know how many retry create index does...

But after adding the cluster health everything rock...

Cheers,

On Mar 21, 2:16 pm, Alois Cochard alois.coch...@gmail.com wrote:

Ok,

Was using version 0.14.0, I'll update ASAP and see if problem still
occure.

Thanks a lot !

On Mar 18, 10:37 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

Which version are you using? the create index API will now automatically retry if its blocked. In any case, you can also wait for yellow cluster state.

On Friday, March 18, 2011 at 11:30 AM, Alois Cochard wrote:

Didn't made a search before posting (shame on me) so I found that:
http://groups.google.com/a/elasticsearch.com/group/users/browse_threa...

But... that's look a bit tricky, any advice Shay ?

On Mar 18, 10:28 am, Alois Cochard alois.coch...@gmail.com wrote:

Hey all !

I've some simple integration test (add an entry then get it, or simple
stuff like that) that start a node, from time to time ([pseudo]-
randomly) I've got the following exception:

org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [1/
not recovered from gateway];
at
org.elasticsearch.cluster.block.ClusterBlocks.indexBlockedRaiseException(Cl usterBlocks.java:
112)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.ch eckBlock(TransportCreateIndexAction.java:
65)
at
org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.ch eckBlock(TransportCreateIndexAction.java:
42)
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction
$1.run(TransportMasterNodeOperationAction.java:91)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

Does anyone experimented the same issue ? any idea why this happen ?

Thanks a lot,

Alois Cochardhttp://aloiscochard.blogspot.comhttp://twitter.com/aloiscochard