Yes, we are using the s3 gateway and version 0.9.0.
Would we get better gateway performance by using the fs gateway and a
network file system, like gluster, drbd, or ceph?
On Aug 12, 11:34 am, Shay Banon shay.ba...@elasticsearch.com wrote:
If I remember correctly, you use the s3 gateway, right? It seems like some
primary shards are stuck initializing (basically, recovering from the
gateway). I forgot to ask, which version are you using?-shay.banon
On Thu, Aug 12, 2010 at 9:01 PM, Grant Rodgers gra...@gmail.com wrote:
Fyi, 19 hours later, there is no change in the state of the cluster,
and the logs are still bouncing between created and recovering. We
will delete and recreate the missing indices, but it seems like
recovery should have happened by now. What is the expected recovery
time for a shard with a few hundred thousand documents?Here is the cluster state for reference. The dacbe3f4-ef13-43ba-aa13-
b6457780f145 node had been restarted a few times, and maybe that
caused some shards to get stuck in the relocating state? Still, I'd
expect that at least one shard would have recovered by now."routing_table" : {
"indices" : {
"versions" : {
"shards" : {
"0" : [ {
"state" : "INITIALIZING",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 0,
"index" : "versions"
}, {
"state" : "UNASSIGNED",
"primary" : false,
"node" : null,
"relocating_node" : null,
"shard" : 0,
"index" : "versions"
} ],
"1" : [ {
"state" : "INITIALIZING",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 1,
"index" : "versions"
}, {
"state" : "UNASSIGNED",
"primary" : false,
"node" : null,
"relocating_node" : null,
"shard" : 1,
"index" : "versions"
} ],
"2" : [ {
"state" : "INITIALIZING",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 2,
"index" : "versions"
}, {
"state" : "UNASSIGNED",
"primary" : false,
"node" : null,
"relocating_node" : null,
"shard" : 2,
"index" : "versions"
} ],
"3" : [ {
"state" : "INITIALIZING",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 3,
"index" : "versions"
}, {
"state" : "UNASSIGNED",
"primary" : false,
"node" : null,
"relocating_node" : null,
"shard" : 3,
"index" : "versions"
} ],
"4" : [ {
"state" : "INITIALIZING",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 4,
"index" : "versions"
}, {
"state" : "UNASSIGNED",
"primary" : false,
"node" : null,
"relocating_node" : null,
"shard" : 4,
"index" : "versions"
} ],
"5" : [ {
"state" : "INITIALIZING",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 5,
"index" : "versions"
}, {
"state" : "UNASSIGNED",
"primary" : false,
"node" : null,
"relocating_node" : null,
"shard" : 5,
"index" : "versions"
} ]
}
},
"stats" : {
"shards" : {
"0" : [ {
"state" : "STARTED",
"primary" : false,
"node" : "b411c443-ebdd-416a-ac35-cce5af38fa3f",
"relocating_node" : null,
"shard" : 0,
"index" : "stats"
}, {
"state" : "STARTED",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 0,
"index" : "stats"
}, {
"state" : "INITIALIZING",
"primary" : false,
"node" : "dacbe3f4-ef13-43ba-aa13-b6457780f145",
"relocating_node" : null,
"shard" : 0,
"index" : "stats"
} ],
"1" : [ {
"state" : "STARTED",
"primary" : false,
"node" : "b411c443-ebdd-416a-ac35-cce5af38fa3f",
"relocating_node" : null,
"shard" : 1,
"index" : "stats"
}, {
"state" : "STARTED",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 1,
"index" : "stats"
}, {
"state" : "INITIALIZING",
"primary" : false,
"node" : "dacbe3f4-ef13-43ba-aa13-b6457780f145",
"relocating_node" : null,
"shard" : 1,
"index" : "stats"
} ],
"2" : [ {
"state" : "STARTED",
"primary" : false,
"node" : "b411c443-ebdd-416a-ac35-cce5af38fa3f",
"relocating_node" : null,
"shard" : 2,
"index" : "stats"
}, {
"state" : "STARTED",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 2,
"index" : "stats"
}, {
"state" : "INITIALIZING",
"primary" : false,
"node" : "dacbe3f4-ef13-43ba-aa13-b6457780f145",
"relocating_node" : null,
"shard" : 2,
"index" : "stats"
} ],
"3" : [ {
"state" : "STARTED",
"primary" : false,
"node" : "b411c443-ebdd-416a-ac35-cce5af38fa3f",
"relocating_node" : null,
"shard" : 3,
"index" : "stats"
}, {
"state" : "STARTED",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 3,
"index" : "stats"
}, {
"state" : "INITIALIZING",
"primary" : false,
"node" : "dacbe3f4-ef13-43ba-aa13-b6457780f145",
"relocating_node" : null,
"shard" : 3,
"index" : "stats"
} ],
"4" : [ {
"state" : "STARTED",
"primary" : false,
"node" : "b411c443-ebdd-416a-ac35-cce5af38fa3f",
"relocating_node" : null,
"shard" : 4,
"index" : "stats"
}, {
"state" : "STARTED",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 4,
"index" : "stats"
}, {
"state" : "INITIALIZING",
"primary" : false,
"node" : "dacbe3f4-ef13-43ba-aa13-b6457780f145",
"relocating_node" : null,
"shard" : 4,
"index" : "stats"
} ],
"5" : [ {
"state" : "STARTED",
"primary" : false,
"node" : "b411c443-ebdd-416a-ac35-cce5af38fa3f",
"relocating_node" : null,
"shard" : 5,
"index" : "stats"
}, {
"state" : "STARTED",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 5,
"index" : "stats"
}, {
"state" : "INITIALIZING",
"primary" : false,
"node" : "dacbe3f4-ef13-43ba-aa13-b6457780f145",
"relocating_node" : null,
"shard" : 5,
"index" : "stats"
} ]
}
},
"revisions" : {
"shards" : {
"0" : [ {
"state" : "INITIALIZING",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 0,
"index" : "revisions"
}, {
"state" : "UNASSIGNED",
"primary" : false,
"node" : null,
"relocating_node" : null,
"shard" : 0,
"index" : "revisions"
} ],
"1" : [ {
"state" : "INITIALIZING",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : null,
"shard" : 1,
"index" : "revisions"
}, {
"state" : "UNASSIGNED",
"primary" : false,
"node" : null,
"relocating_node" : null,
"shard" : 1,
"index" : "revisions"
} ],
"2" : [ {
"state" : "INITIALIZING",
"primary" : false,
"node" : "b411c443-ebdd-416a-ac35-cce5af38fa3f",
"relocating_node" : null,
"shard" : 2,
"index" : "revisions"
}, {
"state" : "RELOCATING",
"primary" : true,
"node" : "a4eada23-ab66-4ab5-bf20-8de09e25a6a3",
"relocating_node" : "dacbe3f4-ef13-43ba-aa13-
b6457780f145",
"shard" : 2,
"index" : "revisions"
} ],
"3" : [ {...
read more »