Setting up ElasticSearch on Amazon

Elastic Stack Elasticsearch
1

Hi,

I am fairly new to using elasticsearch. I read most of the tutorials
and presentations available over the web. However I am still not sure
about this one thing.

In conf file I set up my cluster as shown on the website. The gateway
I am using is s3. I setup my elasticsearch in /usr/local/ and took and
AMI image of it, so next time I create a node it has elasticsearch on
it with my specified configurations. Also in my conf file value for
path to store my data is an EBS which gets mounted as soon as node is
created, for persistence data.

How would elasticsearch look for data if I use s3 gateway and store my
data on EBS volume

Now what I do not understand is, if I am storing my data in EBS /mnt/
storage/ and using s3 gateway. In the event when the
cluster is comes down and is recovering, will the nodes first check
their local storage or will it start downloading data from s3. Data is
there in both the places.

Let me know any comments for this kind of setup.

Thanks!!

2

If you use s3, then the local data stored (on EBS in your case) will be
copied to s3 and then resync'ed on cluster restart. I suggest you stick to
using the local gateway with EBS, and not use s3.

On Thu, Apr 12, 2012 at 10:20 PM, kinesh kxs2624@rit.edu wrote:

Hi,

I am fairly new to using elasticsearch. I read most of the tutorials
and presentations available over the web. However I am still not sure
about this one thing.

In conf file I set up my cluster as shown on the website. The gateway
I am using is s3. I setup my elasticsearch in /usr/local/ and took and
AMI image of it, so next time I create a node it has elasticsearch on
it with my specified configurations. Also in my conf file value for
path to store my data is an EBS which gets mounted as soon as node is
created, for persistence data.

How would elasticsearch look for data if I use s3 gateway and store my
data on EBS volume

Now what I do not understand is, if I am storing my data in EBS /mnt/
storage/ and using s3 gateway. In the event when the
cluster is comes down and is recovering, will the nodes first check
their local storage or will it start downloading data from s3. Data is
there in both the places.

Let me know any comments for this kind of setup.

Thanks!!

3

Thanks Shay!!

On Apr 13, 5:30 am, Shay Banon kim...@gmail.com wrote:

If you use s3, then the local data stored (on EBS in your case) will be
copied to s3 and then resync'ed on cluster restart. I suggest you stick to
using the local gateway with EBS, and not use s3.

On Thu, Apr 12, 2012 at 10:20 PM, kinesh kxs2...@rit.edu wrote:

Hi,

I am fairly new to using elasticsearch. I read most of the tutorials
and presentations available over the web. However I am still not sure
about this one thing.

In conf file I set up my cluster as shown on the website. The gateway
I am using is s3. I setup my elasticsearch in /usr/local/ and took and
AMI image of it, so next time I create a node it has elasticsearch on
it with my specified configurations. Also in my conf file value for
path to store my data is an EBS which gets mounted as soon as node is
created, for persistence data.

How would elasticsearch look for data if I use s3 gateway and store my
data on EBS volume

Now what I do not understand is, if I am storing my data in EBS /mnt/
storage/ and using s3 gateway. In the event when the
cluster is comes down and is recovering, will the nodes first check
their local storage or will it start downloading data from s3. Data is
there in both the places.

Let me know any comments for this kind of setup.

Thanks!!