Elasticsearch on EC2 Tokyo


(md-2) #1

Hi,

I have successfully tested elasticsearch on EC2 by following this
tutorial,
with only one exception of using local(EBS) instead of S3.
http://www.elasticsearch.org/tutorials/2011/08/22/elasticsearch-on-ec2.html

So I tried on my region, Tokyo, with the following configuration,
simply adding 1 line, "cloud.aws.ec2.endpoint".

======================================
cluster.name: es-demo

cloud:
aws:
access_key: AKIAJ6JLMLY5FH5DCM2Q
secret_key: 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

cloud.aws.ec2.endpoint: ec2.ap-northeast-1.amazonaws.com

discovery:
type: ec2

But it fails with an exception(log below, sorry if it gets hard to
read).
Can you suggest me how this should get done?

node1(master, 10.146.71.212)

[2012-04-17 07:59:35,352][INFO ][node ] [Smythe,
Alistair] {0.19.2}[19946]: started
[2012-04-17 08:00:00,854][TRACE][discovery.zen.ping.unicast] [Smythe,
Alistair] [1] failed to connect to [#cloud-i-d1ba2ed1-0][inet[/
10.146.26.73:9300]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/
10.146.26.73:9300]] connect_timeout[30s]
at
org.elasticsearch.transport.netty.NettyTransport.connectToChannelsLight(NettyTransport.java:
533)
at
org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:
499)
at
org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:
477)
at
org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:
132)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing
$3.run(UnicastZenPing.java:271)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
1110)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.net.ConnectException: connection timed out
at
org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink
$Boss.processConnectTimeout(NioClientSocketPipelineSink.java:387)
at
org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSink
$Boss.run(NioClientSocketPipelineSink.java:291)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:
102)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker
$1.run(DeadLockProofWorker.java:42)
... 3 more

node2(10.146.26.73)

[2012-04-17 08:02:45,736][INFO ][node ] [Crime-
Buster] {0.19.2}[19799]: initializing ...
[2012-04-17 08:02:45,749][INFO ][plugins ] [Crime-
Buster] loaded [cloud-aws], sites []
[2012-04-17 08:02:46,863][DEBUG][discovery.zen.ping.multicast] [Crime-
Buster] using group [224.2.2.4], with port [54328], ttl [3], and
address [null]
[2012-04-17 08:02:46,866][DEBUG][discovery.zen.ping.unicast] [Crime-
Buster] using initial hosts [], with concurrent_connects [10]
[2012-04-17 08:02:46,867][DEBUG][discovery.ec2 ] [Crime-
Buster] using ping.timeout [3s]
[2012-04-17 08:02:46,872][DEBUG][discovery.zen.elect ] [Crime-
Buster] using minimum_master_nodes [-1]
[2012-04-17 08:02:46,873][DEBUG][discovery.zen.fd ] [Crime-
Buster] [master] uses ping_interval [1s], ping_timeout [30s],
ping_retries [3]
[2012-04-17 08:02:46,875][DEBUG][discovery.zen.fd ] [Crime-
Buster] [node ] uses ping_interval [1s], ping_timeout [30s],
ping_retries [3]
[2012-04-17 08:02:47,116][DEBUG][discovery.ec2 ] [Crime-
Buster] using host_type [PRIVATE_IP], tags [{}], groups [[]] with
any_group [true], availability_zones [[]]
[2012-04-17 08:02:48,028][DEBUG][gateway.local ] [Crime-
Buster] using initial_shards [quorum], list_timeout [30s]
[2012-04-17 08:02:48,251][DEBUG][gateway.local.state.shards] [Crime-
Buster] took 0s to load started shards state
[2012-04-17 08:02:48,253][DEBUG][gateway.local.state.meta ] [Crime-
Buster] took 0s to load state
[2012-04-17 08:02:48,256][INFO ][node ] [Crime-
Buster] {0.19.2}[19799]: initialized
[2012-04-17 08:02:48,256][INFO ][node ] [Crime-
Buster] {0.19.2}[19799]: starting ...
[2012-04-17 08:02:48,340][INFO ][transport ] [Crime-
Buster] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address
{inet[/10.146.26.73:9300]}
[2012-04-17 08:02:48,345][TRACE][discovery ] [Crime-
Buster] waiting for 30s for the initial state to be set by the
discovery
[2012-04-17 08:02:48,678][TRACE][discovery.ec2 ] [Crime-
Buster] building dynamic unicast discovery nodes...
[2012-04-17 08:02:48,690][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-efb82cef, address 10.146.71.212, transport_address
inet[/10.146.71.212:9300]
[2012-04-17 08:02:48,693][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-d1ba2ed1, address 10.146.26.73, transport_address
inet[/10.146.26.73:9300]
[2012-04-17 08:02:48,693][DEBUG][discovery.ec2 ] [Crime-
Buster] using dynamic discovery nodes [[#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]], [#cloud-i-d1ba2ed1-0][inet[/10.146.26.73:9300]]]
[2012-04-17 08:02:48,747][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]
[2012-04-17 08:02:48,752][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]
[2012-04-17 08:02:48,785][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]: [ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]], master [[Smythe,
Alistair][hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]],
cluster_name[es-demo]}]
[2012-04-17 08:02:48,788][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]:
[ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]}]
[2012-04-17 08:02:50,303][TRACE][discovery.ec2 ] [Crime-
Buster] building dynamic unicast discovery nodes...
[2012-04-17 08:02:50,311][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-efb82cef, address 10.146.71.212, transport_address
inet[/10.146.71.212:9300]
[2012-04-17 08:02:50,332][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-d1ba2ed1, address 10.146.26.73, transport_address
inet[/10.146.26.73:9300]
[2012-04-17 08:02:50,333][DEBUG][discovery.ec2 ] [Crime-
Buster] using dynamic discovery nodes [[#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]], [#cloud-i-d1ba2ed1-0][inet[/10.146.26.73:9300]]]
[2012-04-17 08:02:50,333][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]
[2012-04-17 08:02:50,333][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]
[2012-04-17 08:02:50,335][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]:
[ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]}]
[2012-04-17 08:02:50,335][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]: [ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]], master [[Smythe,
Alistair][hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]],
cluster_name[es-demo]}]
[2012-04-17 08:02:51,979][TRACE][discovery.ec2 ] [Crime-
Buster] building dynamic unicast discovery nodes...
[2012-04-17 08:02:51,986][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-efb82cef, address 10.146.71.212, transport_address
inet[/10.146.71.212:9300]
[2012-04-17 08:02:51,990][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-d1ba2ed1, address 10.146.26.73, transport_address
inet[/10.146.26.73:9300]
[2012-04-17 08:02:51,990][DEBUG][discovery.ec2 ] [Crime-
Buster] using dynamic discovery nodes [[#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]], [#cloud-i-d1ba2ed1-0][inet[/10.146.26.73:9300]]]
[2012-04-17 08:02:51,990][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]
[2012-04-17 08:02:51,991][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]
[2012-04-17 08:02:51,993][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]:
[ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]}]
[2012-04-17 08:02:51,998][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]: [ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]], master [[Smythe,
Alistair][hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]],
cluster_name[es-demo]}]
[2012-04-17 08:02:51,999][DEBUG][discovery.ec2 ] [Crime-
Buster] ping responses:
--> target [[Smythe, Alistair][hoUJvK0STou7tW_B9oXnkw][inet[/
10.146.71.212:9300]]], master [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]]
[2012-04-17 08:02:55,026][INFO ][discovery.ec2 ] [Crime-
Buster] failed to send join request to master [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]], reason
[org.elasticsearch.ElasticSearchTimeoutException: Timeout waiting for
task.]
[2012-04-17 08:02:55,026][TRACE][discovery.ec2 ] [Crime-
Buster] detailed failed reason
org.elasticsearch.ElasticSearchTimeoutException: Timeout waiting for
task.
at
org.elasticsearch.transport.PlainTransportFuture.txGet(PlainTransportFuture.java:
63)
at
org.elasticsearch.transport.PlainTransportFuture.txGet(PlainTransportFuture.java:
35)
at
org.elasticsearch.discovery.zen.membership.MembershipAction.sendJoinRequestBlocking(MembershipAction.java:
84)
at
org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery.java:
321)
at org.elasticsearch.discovery.zen.ZenDiscovery.access
$500(ZenDiscovery.java:69)
at org.elasticsearch.discovery.zen.ZenDiscovery
$1.run(ZenDiscovery.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
1110)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)

Thanks,
Takenori


(md-2) #2

Oops, I left access/secret key.

It's just being invalidated.

On 4月17日, 午後5:09, MD takenori.s...@gmail.com wrote:

Hi,

I have successfully tested elasticsearch on EC2 by following this
tutorial,
with only one exception of using local(EBS) instead of S3.http://www.elasticsearch.org/tutorials/2011/08/22/elasticsearch-on-ec...

So I tried on my region, Tokyo, with the following configuration,
simply adding 1 line, "cloud.aws.ec2.endpoint".

======================================
cluster.name: es-demo

cloud:
aws:
access_key: AKIAJ6JLMLY5FH5DCM2Q
secret_key: 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

cloud.aws.ec2.endpoint: ec2.ap-northeast-1.amazonaws.com

discovery:
type: ec2

But it fails with an exception(log below, sorry if it gets hard to
read).
Can you suggest me how this should get done?

node1(master, 10.146.71.212)

[2012-04-17 07:59:35,352][INFO ][node ] [Smythe,
Alistair] {0.19.2}[19946]: started
[2012-04-17 08:00:00,854][TRACE][discovery.zen.ping.unicast] [Smythe,
Alistair] [1] failed to connect to [#cloud-i-d1ba2ed1-0][inet[/
10.146.26.73:9300]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/
10.146.26.73:9300]] connect_timeout[30s]
at
org.elasticsearch.transport.netty.NettyTransport.connectToChannelsLight(Net tyTransport.java:
533)
at
org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTranspo rt.java:
499)
at
org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTr ansport.java:
477)
at
org.elasticsearch.transport.TransportService.connectToNodeLight(TransportSe rvice.java:
132)
at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing
$3.run(UnicastZenPing.java:271)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
1110)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.net.ConnectException: connection timed out
at
org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSi nk
$Boss.processConnectTimeout(NioClientSocketPipelineSink.java:387)
at
org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSi nk
$Boss.run(NioClientSocketPipelineSink.java:291)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenami ngRunnable.java:
102)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker
$1.run(DeadLockProofWorker.java:42)
... 3 more

node2(10.146.26.73)

[2012-04-17 08:02:45,736][INFO ][node ] [Crime-
Buster] {0.19.2}[19799]: initializing ...
[2012-04-17 08:02:45,749][INFO ][plugins ] [Crime-
Buster] loaded [cloud-aws], sites []
[2012-04-17 08:02:46,863][DEBUG][discovery.zen.ping.multicast] [Crime-
Buster] using group [224.2.2.4], with port [54328], ttl [3], and
address [null]
[2012-04-17 08:02:46,866][DEBUG][discovery.zen.ping.unicast] [Crime-
Buster] using initial hosts [], with concurrent_connects [10]
[2012-04-17 08:02:46,867][DEBUG][discovery.ec2 ] [Crime-
Buster] using ping.timeout [3s]
[2012-04-17 08:02:46,872][DEBUG][discovery.zen.elect ] [Crime-
Buster] using minimum_master_nodes [-1]
[2012-04-17 08:02:46,873][DEBUG][discovery.zen.fd ] [Crime-
Buster] [master] uses ping_interval [1s], ping_timeout [30s],
ping_retries [3]
[2012-04-17 08:02:46,875][DEBUG][discovery.zen.fd ] [Crime-
Buster] [node ] uses ping_interval [1s], ping_timeout [30s],
ping_retries [3]
[2012-04-17 08:02:47,116][DEBUG][discovery.ec2 ] [Crime-
Buster] using host_type [PRIVATE_IP], tags [{}], groups [[]] with
any_group [true], availability_zones [[]]
[2012-04-17 08:02:48,028][DEBUG][gateway.local ] [Crime-
Buster] using initial_shards [quorum], list_timeout [30s]
[2012-04-17 08:02:48,251][DEBUG][gateway.local.state.shards] [Crime-
Buster] took 0s to load started shards state
[2012-04-17 08:02:48,253][DEBUG][gateway.local.state.meta ] [Crime-
Buster] took 0s to load state
[2012-04-17 08:02:48,256][INFO ][node ] [Crime-
Buster] {0.19.2}[19799]: initialized
[2012-04-17 08:02:48,256][INFO ][node ] [Crime-
Buster] {0.19.2}[19799]: starting ...
[2012-04-17 08:02:48,340][INFO ][transport ] [Crime-
Buster] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address
{inet[/10.146.26.73:9300]}
[2012-04-17 08:02:48,345][TRACE][discovery ] [Crime-
Buster] waiting for 30s for the initial state to be set by the
discovery
[2012-04-17 08:02:48,678][TRACE][discovery.ec2 ] [Crime-
Buster] building dynamic unicast discovery nodes...
[2012-04-17 08:02:48,690][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-efb82cef, address 10.146.71.212, transport_address
inet[/10.146.71.212:9300]
[2012-04-17 08:02:48,693][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-d1ba2ed1, address 10.146.26.73, transport_address
inet[/10.146.26.73:9300]
[2012-04-17 08:02:48,693][DEBUG][discovery.ec2 ] [Crime-
Buster] using dynamic discovery nodes [[#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]], [#cloud-i-d1ba2ed1-0][inet[/10.146.26.73:9300]]]
[2012-04-17 08:02:48,747][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]
[2012-04-17 08:02:48,752][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]
[2012-04-17 08:02:48,785][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]: [ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]], master [[Smythe,
Alistair][hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]],
cluster_name[es-demo]}]
[2012-04-17 08:02:48,788][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]:
[ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]}]
[2012-04-17 08:02:50,303][TRACE][discovery.ec2 ] [Crime-
Buster] building dynamic unicast discovery nodes...
[2012-04-17 08:02:50,311][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-efb82cef, address 10.146.71.212, transport_address
inet[/10.146.71.212:9300]
[2012-04-17 08:02:50,332][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-d1ba2ed1, address 10.146.26.73, transport_address
inet[/10.146.26.73:9300]
[2012-04-17 08:02:50,333][DEBUG][discovery.ec2 ] [Crime-
Buster] using dynamic discovery nodes [[#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]], [#cloud-i-d1ba2ed1-0][inet[/10.146.26.73:9300]]]
[2012-04-17 08:02:50,333][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]
[2012-04-17 08:02:50,333][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]
[2012-04-17 08:02:50,335][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]:
[ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]}]
[2012-04-17 08:02:50,335][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]: [ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]], master [[Smythe,
Alistair][hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]],
cluster_name[es-demo]}]
[2012-04-17 08:02:51,979][TRACE][discovery.ec2 ] [Crime-
Buster] building dynamic unicast discovery nodes...
[2012-04-17 08:02:51,986][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-efb82cef, address 10.146.71.212, transport_address
inet[/10.146.71.212:9300]
[2012-04-17 08:02:51,990][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-d1ba2ed1, address 10.146.26.73, transport_address
inet[/10.146.26.73:9300]
[2012-04-17 08:02:51,990][DEBUG][discovery.ec2 ] [Crime-
Buster] using dynamic discovery nodes [[#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]], [#cloud-i-d1ba2ed1-0][inet[/10.146.26.73:9300]]]
[2012-04-17 08:02:51,990][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]
[2012-04-17 08:02:51,991][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]
[2012-04-17 08:02:51,993][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]:
[ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]}]
[2012-04-17 08:02:51,998][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]: [ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]], master [[Smythe,
Alistair][hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]],
cluster_name[es-demo]}]
[2012-04-17 08:02:51,999][DEBUG][discovery.ec2 ] [Crime-
Buster] ping responses:
--> target [[Smythe, Alistair][hoUJvK0STou7tW_B9oXnkw][inet[/
10.146.71.212:9300]]], master [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]]
[2012-04-17 08:02:55,026][INFO ][discovery.ec2 ] [Crime-
Buster] failed to send join request to master [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]], reason
[org.elasticsearch.ElasticSearchTimeoutException: Timeout waiting for
task.]
[2012-04-17 08:02:55,026][TRACE][discovery.ec2 ] [Crime-
Buster] detailed failed reason
org.elasticsearch.ElasticSearchTimeoutException: Timeout waiting for
task.
at
org.elasticsearch.transport.PlainTransportFuture.txGet(PlainTransportFuture .java:
63)
at
org.elasticsearch.transport.PlainTransportFuture.txGet(PlainTransportFuture .java:
35)
at
org.elasticsearch.discovery.zen.membership.MembershipAction.sendJoinRequest Blocking(MembershipAction.java:
84)
at
org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery .java:
321)
at org.elasticsearch.discovery.zen.ZenDiscovery.access
$500(ZenDiscovery.java:69)
at org.elasticsearch.discovery.zen.ZenDiscovery
$1.run(ZenDiscovery.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
1110)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)

Thanks,
Takenori


(md-2) #3

Not sure, but it worked after relaunching new instances!

Thanks for the great framework!

On Tuesday, April 17, 2012 5:19:26 PM UTC+9, Sato Takenori wrote:

Oops, I left access/secret key.

It's just being invalidated.

On 4月17日, 午後5:09, MD takenori.s...@gmail.com wrote:

Hi,

I have successfully tested elasticsearch on EC2 by following this
tutorial,
with only one exception of using local(EBS) instead of S3.
http://www.elasticsearch.org/tutorials/2011/08/22/elasticsearch-on-ec...

So I tried on my region, Tokyo, with the following configuration,
simply adding 1 line, "cloud.aws.ec2.endpoint".

======================================
cluster.name: es-demo

cloud:
aws:
access_key: AKIAJ6JLMLY5FH5DCM2Q
secret_key: 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

cloud.aws.ec2.endpoint: ec2.ap-northeast-1.amazonaws.com

discovery:
type: ec2

But it fails with an exception(log below, sorry if it gets hard to
read).
Can you suggest me how this should get done?

node1(master, 10.146.71.212)

[2012-04-17 07:59:35,352][INFO ][node ] [Smythe,
Alistair] {0.19.2}[19946]: started
[2012-04-17 08:00:00,854][TRACE][discovery.zen.ping.unicast] [Smythe,
Alistair] [1] failed to connect to [#cloud-i-d1ba2ed1-0][inet[/
10.146.26.73:9300]]
org.elasticsearch.transport.ConnectTransportException: [][inet[/
10.146.26.73:9300]] connect_timeout[30s]
at

org.elasticsearch.transport.netty.NettyTransport.connectToChannelsLight(Net
tyTransport.java:

  1. at

org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTranspo
rt.java:

  1. at

org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTr
ansport.java:

  1. at

org.elasticsearch.transport.TransportService.connectToNodeLight(TransportSe
rvice.java:

  1. at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing
    $3.run(UnicastZenPing.java:271)
    at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:

  1. at java.util.concurrent.ThreadPoolExecutor
    $Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:679)
    Caused by: java.net.ConnectException: connection timed out
    at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSi
nk

$Boss.processConnectTimeout(NioClientSocketPipelineSink.java:387)
at

org.elasticsearch.common.netty.channel.socket.nio.NioClientSocketPipelineSi
nk

$Boss.run(NioClientSocketPipelineSink.java:291)
at

org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenami
ngRunnable.java:

  1. at
    org.elasticsearch.common.netty.util.internal.DeadLockProofWorker
    $1.run(DeadLockProofWorker.java:42)
    ... 3 more

node2(10.146.26.73)

[2012-04-17 08:02:45,736][INFO ][node ] [Crime-
Buster] {0.19.2}[19799]: initializing ...
[2012-04-17 08:02:45,749][INFO ][plugins ] [Crime-
Buster] loaded [cloud-aws], sites []
[2012-04-17 08:02:46,863][DEBUG][discovery.zen.ping.multicast] [Crime-
Buster] using group [224.2.2.4], with port [54328], ttl [3], and
address [null]
[2012-04-17 08:02:46,866][DEBUG][discovery.zen.ping.unicast] [Crime-
Buster] using initial hosts [], with concurrent_connects [10]
[2012-04-17 08:02:46,867][DEBUG][discovery.ec2 ] [Crime-
Buster] using ping.timeout [3s]
[2012-04-17 08:02:46,872][DEBUG][discovery.zen.elect ] [Crime-
Buster] using minimum_master_nodes [-1]
[2012-04-17 08:02:46,873][DEBUG][discovery.zen.fd ] [Crime-
Buster] [master] uses ping_interval [1s], ping_timeout [30s],
ping_retries [3]
[2012-04-17 08:02:46,875][DEBUG][discovery.zen.fd ] [Crime-
Buster] [node ] uses ping_interval [1s], ping_timeout [30s],
ping_retries [3]
[2012-04-17 08:02:47,116][DEBUG][discovery.ec2 ] [Crime-
Buster] using host_type [PRIVATE_IP], tags [{}], groups [[]] with
any_group [true], availability_zones [[]]
[2012-04-17 08:02:48,028][DEBUG][gateway.local ] [Crime-
Buster] using initial_shards [quorum], list_timeout [30s]
[2012-04-17 08:02:48,251][DEBUG][gateway.local.state.shards] [Crime-
Buster] took 0s to load started shards state
[2012-04-17 08:02:48,253][DEBUG][gateway.local.state.meta ] [Crime-
Buster] took 0s to load state
[2012-04-17 08:02:48,256][INFO ][node ] [Crime-
Buster] {0.19.2}[19799]: initialized
[2012-04-17 08:02:48,256][INFO ][node ] [Crime-
Buster] {0.19.2}[19799]: starting ...
[2012-04-17 08:02:48,340][INFO ][transport ] [Crime-
Buster] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address
{inet[/10.146.26.73:9300]}
[2012-04-17 08:02:48,345][TRACE][discovery ] [Crime-
Buster] waiting for 30s for the initial state to be set by the
discovery
[2012-04-17 08:02:48,678][TRACE][discovery.ec2 ] [Crime-
Buster] building dynamic unicast discovery nodes...
[2012-04-17 08:02:48,690][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-efb82cef, address 10.146.71.212, transport_address
inet[/10.146.71.212:9300]
[2012-04-17 08:02:48,693][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-d1ba2ed1, address 10.146.26.73, transport_address
inet[/10.146.26.73:9300]
[2012-04-17 08:02:48,693][DEBUG][discovery.ec2 ] [Crime-
Buster] using dynamic discovery nodes [[#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]], [#cloud-i-d1ba2ed1-0][inet[/10.146.26.73:9300]]]
[2012-04-17 08:02:48,747][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]
[2012-04-17 08:02:48,752][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]
[2012-04-17 08:02:48,785][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]: [ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]], master [[Smythe,
Alistair][hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]],
cluster_name[es-demo]}]
[2012-04-17 08:02:48,788][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]:
[ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]}]
[2012-04-17 08:02:50,303][TRACE][discovery.ec2 ] [Crime-
Buster] building dynamic unicast discovery nodes...
[2012-04-17 08:02:50,311][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-efb82cef, address 10.146.71.212, transport_address
inet[/10.146.71.212:9300]
[2012-04-17 08:02:50,332][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-d1ba2ed1, address 10.146.26.73, transport_address
inet[/10.146.26.73:9300]
[2012-04-17 08:02:50,333][DEBUG][discovery.ec2 ] [Crime-
Buster] using dynamic discovery nodes [[#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]], [#cloud-i-d1ba2ed1-0][inet[/10.146.26.73:9300]]]
[2012-04-17 08:02:50,333][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]
[2012-04-17 08:02:50,333][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]
[2012-04-17 08:02:50,335][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]:
[ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]}]
[2012-04-17 08:02:50,335][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]: [ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]], master [[Smythe,
Alistair][hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]],
cluster_name[es-demo]}]
[2012-04-17 08:02:51,979][TRACE][discovery.ec2 ] [Crime-
Buster] building dynamic unicast discovery nodes...
[2012-04-17 08:02:51,986][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-efb82cef, address 10.146.71.212, transport_address
inet[/10.146.71.212:9300]
[2012-04-17 08:02:51,990][TRACE][discovery.ec2 ] [Crime-
Buster] adding i-d1ba2ed1, address 10.146.26.73, transport_address
inet[/10.146.26.73:9300]
[2012-04-17 08:02:51,990][DEBUG][discovery.ec2 ] [Crime-
Buster] using dynamic discovery nodes [[#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]], [#cloud-i-d1ba2ed1-0][inet[/10.146.26.73:9300]]]
[2012-04-17 08:02:51,990][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]
[2012-04-17 08:02:51,991][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] connecting to [Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]
[2012-04-17 08:02:51,993][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]:
[ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]},
ping_response{target [[Crime-Buster][OqfkGUtZRQO4EwPWSoDT_A][inet[/
10.146.26.73:9300]]], master [null], cluster_name[es-demo]}]
[2012-04-17 08:02:51,998][TRACE][discovery.zen.ping.unicast] [Crime-
Buster] [1] received response from [#cloud-i-efb82cef-0][inet[/
10.146.71.212:9300]]: [ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Crime-Buster]
[OqfkGUtZRQO4EwPWSoDT_A][inet[/10.146.26.73:9300]]], master [null],
cluster_name[es-demo]}, ping_response{target [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]], master [[Smythe,
Alistair][hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]],
cluster_name[es-demo]}]
[2012-04-17 08:02:51,999][DEBUG][discovery.ec2 ] [Crime-
Buster] ping responses:
--> target [[Smythe, Alistair][hoUJvK0STou7tW_B9oXnkw][inet[/
10.146.71.212:9300]]], master [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]]
[2012-04-17 08:02:55,026][INFO ][discovery.ec2 ] [Crime-
Buster] failed to send join request to master [[Smythe, Alistair]
[hoUJvK0STou7tW_B9oXnkw][inet[/10.146.71.212:9300]]], reason
[org.elasticsearch.ElasticSearchTimeoutException: Timeout waiting for
task.]
[2012-04-17 08:02:55,026][TRACE][discovery.ec2 ] [Crime-
Buster] detailed failed reason
org.elasticsearch.ElasticSearchTimeoutException: Timeout waiting for
task.
at

org.elasticsearch.transport.PlainTransportFuture.txGet(PlainTransportFuture
.java:

  1. at 
    

org.elasticsearch.transport.PlainTransportFuture.txGet(PlainTransportFuture
.java:

  1. at 
    

org.elasticsearch.discovery.zen.membership.MembershipAction.sendJoinRequest
Blocking(MembershipAction.java:

  1. at 
    

org.elasticsearch.discovery.zen.ZenDiscovery.innterJoinCluster(ZenDiscovery
.java:

  1. at org.elasticsearch.discovery.zen.ZenDiscovery.access
    $500(ZenDiscovery.java:69)
    at org.elasticsearch.discovery.zen.ZenDiscovery
    $1.run(ZenDiscovery.java:266)
    at

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:

  1. at java.util.concurrent.ThreadPoolExecutor
    $Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:679)

Thanks,
Takenori


(system) #4