Es newbie trying to understand adding a node on ec2

Elastic Stack Elasticsearch
1

Hi,

I've setup es on an ec2 micro instance and have it working, sending
json back and
forth is pretty cool.

I've gone through the docs and googled around but am still unable to
understand the following.

  1. I'm currently indexing user transaction records and would like to
    add another node on another ec2 instance. How do I do this? A step by
    step walkthru for an es&ec2 newbie would be helpful.

  2. If in addition to this I also wanted to index the Twitter River,
    how would I do that i.e. would I need a separate es server instance on
    a separate ec2 box (if so how would it play as a node too) or would I
    use multi-tenancy to do this, if so how? Whilst on the Twitter River
    subject I notice you use username, password auth and the twitter folks
    want everyone to start using oauth, if one got the oauth token out-of-
    band than one would only need to send a signed url from within ES, So
    if i built the signed url is there a way i can give it to ES?

  3. Is there a document someplace that provides a deeper explanation
    into shards & replicas?

Thanks

2

Hi,

So I added a second ec2 instance, installed elasticsearch on it and
than on instance 1 I brought down elasticsearch, changed the config/
e...yml file to add ec2 discovery as per
http://www.elasticsearch.com/docs/elasticsearch/cloud/aws/ec2_discovery/
I than brought elasticsearch back up on instance 1. I did the same on
instance 2 i.e. edited the config/e..yml file to reflect ec2 discovery
and brought elasticsearch up.

My question is, should I not now see that instance2 is another node
and a replica of instance1 and its data directory start filling up.
That however does not seem to be happening. Could someone outline what
I should see/expect, what I might be doing wrong etc.

Inline below for future newbies I've added some notes that might save
them some time

On Jan 7, 10:52 am, molicule molic...@gmail.com wrote:

  1. Is there a document someplace that provides a deeper explanation
    into shards & replicas?

There are 4 places in which to search for the documentation i.e.

  1. The docs on elasticsearch.com
  2. The wiki and issues sections of the github wiki
  3. The blog
  4. This user list

for shards, replicas etc. read GitHub - elastic/elasticsearch: Free and Open, Distributed, RESTful Search Engine

For installing plugins see Documentation suggests deprecated (?) cloud plugin · Issue #582 · elastic/elasticsearch · GitHub

3

For me the minimum elastic search config ( in ./conf/elasticsearch.yml ) I've been able to get it working with is;

cloud: aws: access_key: $AWS_ACCESS_KEY secret_key: $AWS_SECRET_KEY

discovery:
type: ec2

Where $AWS_ACCESS_KEY and $AWS_SECRET_KEY are replaced with their values. This also assumes you're using the default us-east1 datacenter.

You also need to give permission to port 9300 as well, the port used for cluster communication - see docs on ec2-authorize

Once multiple nodes are started, you can see if the cluster is working by accessing http://:9200/_cluster/health - you should see something like "number_of_nodes: 2"

4

Thanks a heap! That was it i.e. I needed to open port 9300 for
10.0.0.0/8 and now _cluster/health is green with 2 nodes

On Jan 9, 5:42 pm, harryf hfue...@gmail.com wrote:

For me the minimum Elasticsearch config ( in ./conf/elasticsearch.yml ) I've
been able to get it working with is;

cloud:
aws:
access_key: $AWS_ACCESS_KEY
secret_key: $AWS_SECRET_KEY

discovery:
type: ec2

Where $AWS_ACCESS_KEY and $AWS_SECRET_KEY are replaced with their values.
This also assumes you're using the default us-east1 datacenter.

You also need to give permission to port 9300 as well, the port used for
cluster communication - see docs on ec2-authorize

Once multiple nodes are started, you can see if the cluster is working by
accessing http://:9200/_cluster/health - you should see something
like "number_of_nodes: 2"

View this message in context:http://elasticsearch-users.115913.n3.nabble.com/es-newbie-trying-to-u...
Sent from the Elasticsearch Users mailing list archive at Nabble.com.