EC2 Configuration

Elastic Stack Elasticsearch
1

I have been using Hazelcast in a dev environment (local laptop) for a couple
months while we develop our application. It is now the point where I want to
bring HC to a QA environment which will be on EC2. Are there any best
practices for configuration on EC2.

I've come across this post which has an EC2 config. It relies on a custom
token being defined when the config is created. I would like to avoid this.
http://groups.google.com/group/hazelcast/msg/69204fd1025d95e7?pli=1

Thanks.

2

Not sure about Hazelcrast, but discovery in elasticsearch is very simple
using the cloud plugin. Regarding what type of machines, it really depends
on the load you are after, but I would probably say the large instances are
a minimum just because of the poor IO on small instances (from what I
heard).

-shay.banon

On Tue, Aug 31, 2010 at 8:05 PM, James Cook jcook@tracermedia.com wrote:

I have been using Hazelcast in a dev environment (local laptop) for a
couple months while we develop our application. It is now the point where I
want to bring HC to a QA environment which will be on EC2. Are there any
best practices for configuration on EC2.

I've come across this post which has an EC2 config. It relies on a custom
token being defined when the config is created. I would like to avoid this.
http://groups.google.com/group/hazelcast/msg/69204fd1025d95e7?pli=1

Thanks.

3

Hi Shay,

Sorry, I posted to the wrong group. :*)

Although I would also like to get info regarding best practices for ES as
well.

My proposed EC2 config is here:

    node.data = true
    network.host = 127.0.0.1
    transport.tcp.port = 9310
    http.enabled = false

    cloud.aws.access_key = XXX
    cloud.aws.secret_key = XXX
    discovery.type = ec2

    gateway.type = s3
    s3.bucket = nep_qa/es_gateway

    index.store.memory.direct=true

I know each of these will depend on the particular application. I am looking
to have a set of scalable instances that will shrink/grow between 2 and 10
instances. That upper limit is just a guess at this point until we can
perform real load testing.

The only question that leaps to my mind is whether there is any benefit to
using a file-based index on servers which take their hard drive with them
when they are shutdown gracefully or disappear unexpectedly.

On Tue, Aug 31, 2010 at 2:26 PM, Shay Banon shay.banon@elasticsearch.comwrote:

Not sure about Hazelcrast, but discovery in elasticsearch is very simple
using the cloud plugin. Regarding what type of machines, it really depends
on the load you are after, but I would probably say the large instances are
a minimum just because of the poor IO on small instances (from what I
heard).

-shay.banon

On Tue, Aug 31, 2010 at 8:05 PM, James Cook jcook@tracermedia.com wrote:

I have been using Hazelcast in a dev environment (local laptop) for a
couple months while we develop our application. It is now the point where I
want to bring HC to a QA environment which will be on EC2. Are there any
best practices for configuration on EC2.

I've come across this post which has an EC2 config. It relies on a custom
token being defined when the config is created. I would like to avoid this.
http://groups.google.com/group/hazelcast/msg/69204fd1025d95e7?pli=1

Thanks.

4

Of course the network.host setting will not work, since its will only use
localhost. You should simply not set it when running on ec2.

file based indices make sense, especially when running on ec2 with s3
gateway, since they can then be reused when performing full recovery (done
automatically by elasticsearch).

-shay.banon

On Tue, Aug 31, 2010 at 9:42 PM, James Cook jcook@tracermedia.com wrote:

Hi Shay,

Sorry, I posted to the wrong group. :*)

Although I would also like to get info regarding best practices for ES as
well.

My proposed EC2 config is here:

    node.data = true
    network.host = 127.0.0.1
    transport.tcp.port = 9310
    http.enabled = false

    cloud.aws.access_key = XXX
    cloud.aws.secret_key = XXX
    discovery.type = ec2

    gateway.type = s3
    s3.bucket = nep_qa/es_gateway

    index.store.memory.direct=true

I know each of these will depend on the particular application. I am
looking to have a set of scalable instances that will shrink/grow between 2
and 10 instances. That upper limit is just a guess at this point until we
can perform real load testing.

The only question that leaps to my mind is whether there is any benefit to
using a file-based index on servers which take their hard drive with them
when they are shutdown gracefully or disappear unexpectedly.

On Tue, Aug 31, 2010 at 2:26 PM, Shay Banon shay.banon@elasticsearch.comwrote:

Not sure about Hazelcrast, but discovery in elasticsearch is very simple
using the cloud plugin. Regarding what type of machines, it really depends
on the load you are after, but I would probably say the large instances are
a minimum just because of the poor IO on small instances (from what I
heard).

-shay.banon

On Tue, Aug 31, 2010 at 8:05 PM, James Cook jcook@tracermedia.comwrote:

I have been using Hazelcast in a dev environment (local laptop) for a
couple months while we develop our application. It is now the point where I
want to bring HC to a QA environment which will be on EC2. Are there any
best practices for configuration on EC2.

I've come across this post which has an EC2 config. It relies on a custom
token being defined when the config is created. I would like to avoid this.
http://groups.google.com/group/hazelcast/msg/69204fd1025d95e7?pli=1

Thanks.