0.8.0 - aws - cloud discovery failing with a jgroups related Exception


(zuzur) #1

Hello,

I'm trying to build a small, aws-based test cluster using the
following configuration:

cloud:
type: amazon
account: XXXXXXXXXXXXXXXXXXX
key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
compute:
type: amazon
discovery:
type: cloud
gateway:
type: cloud
cloud:
container: elasticsearch-data

I get the following error when starting up elasticsearch (that's the
master, there is no other instance currently running it ...)

[07:46:02,194][INFO ][node ] [Bison]
{ElasticSearch/0.8.0}[13396]: Initializing ...
[07:46:02,216][INFO ][plugins ] [Bison] Loaded
[cloud]
[07:46:09,439][INFO ][cloud.compute ] [Bison] Connected to
ec2/xxxxxxxxxxxxxxxxxx compute service
[07:46:09,936][INFO ][cloud.blobstore ] [Bison] Connected to
s3/xxxxxxxxxxxxxxxxxx blob store service
[07:46:10,793][INFO ][node ] [Bison]
{ElasticSearch/0.8.0}[13396]: Initialized
[07:46:10,793][INFO ][node ] [Bison]
{ElasticSearch/0.8.0}[13396]: Starting ...
[07:46:10,883][INFO ][transport ] [Bison]
bound_address[inet[/0:0:0:0:0:0:0:0:9300]], publish_address[inet[/
10.241.70.84:9300]]
[07:46:11,244][WARN ]
[cloud.jclouds.aws.ec2.compute.functions.RunningInstanceToNodeMetadata]
[Bison] no tag parsed from i-xxxxxxxx's groups: [Restricted]
< lots of the same message about other instances we are running >
[07:46:16,713][ERROR][jclouds.compute ] [Bison] nodes,
completed: 3/4, errors: 1, rate: 1453ms/op
java.util.concurrent.ExecutionException:
java.util.NoSuchElementException
at java.util.concurrent.FutureTask
$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:
60)
at org.jclouds.concurrent.ConcurrentUtils
$1.run(ConcurrentUtils.java:69)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.util.NoSuchElementException
at java.util.LinkedHashMap
$LinkedHashIterator.nextEntry(LinkedHashMap.java:374)
at java.util.LinkedHashMap$KeyIterator.next(LinkedHashMap.java:
383)
at
com.google.common.collect.Iterators.getOnlyElement(Iterators.java:271)
at
com.google.common.collect.Iterables.getOnlyElement(Iterables.java:206)
at
org.jclouds.aws.ec2.compute.functions.RunningInstanceToNodeMetadata.getLoginAccountFor(RunningInstanceToNodeMetadata.java:
270)
at
org.jclouds.aws.ec2.compute.functions.RunningInstanceToNodeMetadata.getCredentialsForInstanceWithTag(RunningInstanceToNodeMetadata.java:
181)
at
org.jclouds.aws.ec2.compute.functions.RunningInstanceToNodeMetadata.apply(RunningInstanceToNodeMetadata.java:
157)
at
org.jclouds.aws.ec2.compute.functions.RunningInstanceToNodeMetadata.apply(RunningInstanceToNodeMetadata.java:
61)
at com.google.common.collect.Iterators$8.next(Iterators.java:
730)
at com.google.common.collect.Iterators.addAll(Iterators.java:
330)
at com.google.common.collect.Iterables.addAll(Iterables.java:
249)
at
org.jclouds.aws.ec2.compute.config.EC2ComputeServiceContextModule
$EC2ListNodesStrategy$1.call(EC2ComputeServiceContextModule.java:206)
at
org.jclouds.aws.ec2.compute.config.EC2ComputeServiceContextModule
$EC2ListNodesStrategy$1.call(EC2ComputeServiceContextModule.java:203)
at java.util.concurrent.FutureTask
$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
[07:46:16,714][ERROR][jclouds.compute ] [Bison] nodes,
completed: 3/4, errors: 1, rate: 1454ms/op
java.lang.RuntimeException: nodes, completed: 3/4, errors: 1, rate:
1454ms/op
at
org.jclouds.concurrent.ConcurrentUtils.awaitCompletion(ConcurrentUtils.java:
87)
at
org.jclouds.aws.ec2.compute.config.EC2ComputeServiceContextModule
$EC2ListNodesStrategy.listDetailsOnNodesMatching(EC2ComputeServiceContextModule.java:
213)
at
org.jclouds.aws.ec2.compute.config.EC2ComputeServiceContextModule
$EC2ListNodesStrategy.list(EC2ComputeServiceContextModule.java:191)
at
org.jclouds.compute.internal.BaseComputeService.listNodes(BaseComputeService.java:
228)
at
org.elasticsearch.discovery.cloud.CloudZenPing.buildDynamicNodes(CloudZenPing.java:
68)
at
org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:
172)
at
org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:
154)
at
org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:
130)
at
org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:
112)
at
org.elasticsearch.discovery.zen.ZenDiscovery.broadPingTillMasterResolved(ZenDiscovery.java:
407)
at
org.elasticsearch.discovery.zen.ZenDiscovery.initialJoin(ZenDiscovery.java:
226)
at
org.elasticsearch.discovery.zen.ZenDiscovery.doStart(ZenDiscovery.java:
147)
at
org.elasticsearch.util.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:
80)
at
org.elasticsearch.discovery.DiscoveryService.doStart(DiscoveryService.java:
58)
at
org.elasticsearch.util.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:
80)
at
org.elasticsearch.node.internal.InternalNode.start(InternalNode.java:
176)
at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:
115)
at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:
168)
[07:46:16,715][ERROR][bootstrap ] [Bison]
{ElasticSearch/0.8.0}: Startup Failed ...

  • RuntimeException[error parsing nodes in regions: {us-
    east-1=java.util.concurrent.ExecutionException:
    java.util.NoSuchElementException}]

Apparently, this error has been reported (jgroups)
http://code.google.com/p/jclouds/issues/detail?id=270#c1. Any
workaround ? anyone managed to start 0.8.0 with the cloud plugin on
aws ?

Thanks in advance,

Erwan


(Shay Banon) #2

I am not aware of any workaround for this, I will look into this. For
version 0.9, I hope to write native support for aws and hopefully rackspace.

-shay.banon

On Wed, Jun 9, 2010 at 11:20 AM, Erwan A. earzur@gmail.com wrote:

Hello,

I'm trying to build a small, aws-based test cluster using the
following configuration:

cloud:
type: amazon
account: XXXXXXXXXXXXXXXXXXX
key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
compute:
type: amazon
discovery:
type: cloud
gateway:
type: cloud
cloud:
container: elasticsearch-data

I get the following error when starting up elasticsearch (that's the
master, there is no other instance currently running it ...)

[07:46:02,194][INFO ][node ] [Bison]
{ElasticSearch/0.8.0}[13396]: Initializing ...
[07:46:02,216][INFO ][plugins ] [Bison] Loaded
[cloud]
[07:46:09,439][INFO ][cloud.compute ] [Bison] Connected to
ec2/xxxxxxxxxxxxxxxxxx compute service
[07:46:09,936][INFO ][cloud.blobstore ] [Bison] Connected to
s3/xxxxxxxxxxxxxxxxxx blob store service
[07:46:10,793][INFO ][node ] [Bison]
{ElasticSearch/0.8.0}[13396]: Initialized
[07:46:10,793][INFO ][node ] [Bison]
{ElasticSearch/0.8.0}[13396]: Starting ...
[07:46:10,883][INFO ][transport ] [Bison]
bound_address[inet[/0:0:0:0:0:0:0:0:9300]], publish_address[inet[/
10.241.70.84:9300]]
[07:46:11,244][WARN ]
[cloud.jclouds.aws.ec2.compute.functions.RunningInstanceToNodeMetadata]
[Bison] no tag parsed from i-xxxxxxxx's groups: [Restricted]
< lots of the same message about other instances we are running >
[07:46:16,713][ERROR][jclouds.compute ] [Bison] nodes,
completed: 3/4, errors: 1, rate: 1453ms/op
java.util.concurrent.ExecutionException:
java.util.NoSuchElementException
at java.util.concurrent.FutureTask
$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at

com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:
60)
at org.jclouds.concurrent.ConcurrentUtils
$1.run(ConcurrentUtils.java:69)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.util.NoSuchElementException
at java.util.LinkedHashMap
$LinkedHashIterator.nextEntry(LinkedHashMap.java:374)
at java.util.LinkedHashMap$KeyIterator.next(LinkedHashMap.java:
383)
at
com.google.common.collect.Iterators.getOnlyElement(Iterators.java:271)
at
com.google.common.collect.Iterables.getOnlyElement(Iterables.java:206)
at

org.jclouds.aws.ec2.compute.functions.RunningInstanceToNodeMetadata.getLoginAccountFor(RunningInstanceToNodeMetadata.java:
270)
at

org.jclouds.aws.ec2.compute.functions.RunningInstanceToNodeMetadata.getCredentialsForInstanceWithTag(RunningInstanceToNodeMetadata.java:
181)
at

org.jclouds.aws.ec2.compute.functions.RunningInstanceToNodeMetadata.apply(RunningInstanceToNodeMetadata.java:
157)
at

org.jclouds.aws.ec2.compute.functions.RunningInstanceToNodeMetadata.apply(RunningInstanceToNodeMetadata.java:
61)
at com.google.common.collect.Iterators$8.next(Iterators.java:
730)
at com.google.common.collect.Iterators.addAll(Iterators.java:
330)
at com.google.common.collect.Iterables.addAll(Iterables.java:
249)
at
org.jclouds.aws.ec2.compute.config.EC2ComputeServiceContextModule
$EC2ListNodesStrategy$1.call(EC2ComputeServiceContextModule.java:206)
at
org.jclouds.aws.ec2.compute.config.EC2ComputeServiceContextModule
$EC2ListNodesStrategy$1.call(EC2ComputeServiceContextModule.java:203)
at java.util.concurrent.FutureTask
$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
[07:46:16,714][ERROR][jclouds.compute ] [Bison] nodes,
completed: 3/4, errors: 1, rate: 1454ms/op
java.lang.RuntimeException: nodes, completed: 3/4, errors: 1, rate:
1454ms/op
at

org.jclouds.concurrent.ConcurrentUtils.awaitCompletion(ConcurrentUtils.java:
87)
at
org.jclouds.aws.ec2.compute.config.EC2ComputeServiceContextModule

$EC2ListNodesStrategy.listDetailsOnNodesMatching(EC2ComputeServiceContextModule.java:
213)
at
org.jclouds.aws.ec2.compute.config.EC2ComputeServiceContextModule
$EC2ListNodesStrategy.list(EC2ComputeServiceContextModule.java:191)
at

org.jclouds.compute.internal.BaseComputeService.listNodes(BaseComputeService.java:
228)
at

org.elasticsearch.discovery.cloud.CloudZenPing.buildDynamicNodes(CloudZenPing.java:
68)
at

org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.sendPings(UnicastZenPing.java:
172)
at

org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.ping(UnicastZenPing.java:
154)
at

org.elasticsearch.discovery.zen.ping.ZenPingService.ping(ZenPingService.java:
130)
at

org.elasticsearch.discovery.zen.ping.ZenPingService.pingAndWait(ZenPingService.java:
112)
at

org.elasticsearch.discovery.zen.ZenDiscovery.broadPingTillMasterResolved(ZenDiscovery.java:
407)
at
org.elasticsearch.discovery.zen.ZenDiscovery.initialJoin(ZenDiscovery.java:
226)
at
org.elasticsearch.discovery.zen.ZenDiscovery.doStart(ZenDiscovery.java:
147)
at

org.elasticsearch.util.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:
80)
at
org.elasticsearch.discovery.DiscoveryService.doStart(DiscoveryService.java:
58)
at

org.elasticsearch.util.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:
80)
at
org.elasticsearch.node.internal.InternalNode.start(InternalNode.java:
176)
at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:
115)
at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:
168)
[07:46:16,715][ERROR][bootstrap ] [Bison]
{ElasticSearch/0.8.0}: Startup Failed ...

  • RuntimeException[error parsing nodes in regions: {us-
    east-1=java.util.concurrent.ExecutionException:
    java.util.NoSuchElementException}]

Apparently, this error has been reported (jgroups)
http://code.google.com/p/jclouds/issues/detail?id=270#c1. Any
workaround ? anyone managed to start 0.8.0 with the cloud plugin on
aws ?

Thanks in advance,

Erwan


(Erwan Arzur-2) #3

2010/6/10 Shay Banon shay.banon@elasticsearch.com:

I am not aware of any workaround for this, I will look into this. For
version 0.9, I hope to write native support for aws and hopefully rackspace.
-shay.banon

Thanks, should I file a bug ?

Erwan


(Shay Banon) #4

I am aware of this :), and it will definitely be fixed. I don't think there
is a need to file a bug, its going to work differently in the future anyhow.

On Thu, Jun 10, 2010 at 4:25 PM, Erwan Arzur erwan@arzur.net wrote:

2010/6/10 Shay Banon shay.banon@elasticsearch.com:

I am not aware of any workaround for this, I will look into this. For
version 0.9, I hope to write native support for aws and hopefully
rackspace.
-shay.banon

Thanks, should I file a bug ?

Erwan


(system) #5