We currently have a single Elasticsearch box running 0.17.6 - it's using
the fs gateway type.
We're really liking the tool and it's working great for our uses - now I am
stepping in to deploy a cluster of boxes using 0.18.4. My goal is to:
Have a cluster of nodes running for speed and availability.
Use the S3/Cloudfiles gateway to keep gateway persistent and more
durable with VPS failures.
I have been reading the mailing list/guides/tutorials/etc and have come up
with this plan but have a couple questions.
First the proposed plan:
Flush and shutdown the currently running 0.17.6.
Update the binaries to 0.18.4.
Start the new binaries and make sure they work.
Add another node to the cluster.
Here are my questions:
Because it's currently using the fs gateway, unless they all have access
to that directory of files then the others will not be able to recover from
failure properly. We should have used local gateway for that single box
right?
If I add the new nodes with the S3/Cloudfiles gateway, will they
replicate all of the indexes properly to S3/Cloudfiles? Or do they only add
the items when they're indexed? I have read a ton about people trying to do
that, looks like they need to re-index so that the offsite gateways get
populated - is that correct?
The clustering seems to be pretty magical as far as discovery goes - this
is the config that worked great with Rackspace Cloud
boxes: https://gist.github.com/1390228
Do the boxes have to be on the same subnet to find each other?
I couldn't find any details about that and it all worked flawlessly, but
wondering how I'll add boxes in the future when they run out of IP
addresses that are "nearby". Or if that even matters.
Sorry if I've missed something, I have read everything I can find and I'm
just trying to make sure on my last few questions. I found a lot of
information that appeared to apply to old releases and just wanted to
clarify.
We're really loving the product and I spent much of this weekend adding and
removing nodes to my cluster, dropping and adding all sorts of indexes and
watching it rebalance - very nice work so far.
You say you use Rackspace, and it works, but the config points to AWS
configuration?
There is no rackspace cloudfiles support to act as gateway, only s3.
In any case for 2, you should start with local gateway, its perfectly
fine to use on one node, to many. I don't understand why you used fs
gateway in your one node scenario now.
We currently have a single Elasticsearch box running 0.17.6 - it's using
the fs gateway type.
We're really liking the tool and it's working great for our uses - now I
am stepping in to deploy a cluster of boxes using 0.18.4. My goal is to:
Have a cluster of nodes running for speed and availability.
Use the S3/Cloudfiles gateway to keep gateway persistent and more
durable with VPS failures.
I have been reading the mailing list/guides/tutorials/etc and have come up
with this plan but have a couple questions.
First the proposed plan:
Flush and shutdown the currently running 0.17.6.
Update the binaries to 0.18.4.
Start the new binaries and make sure they work.
Add another node to the cluster.
Here are my questions:
Because it's currently using the fs gateway, unless they all have
access to that directory of files then the others will not be able to
recover from failure properly. We should have used local gateway for that
single box right?
If I add the new nodes with the S3/Cloudfiles gateway, will they
replicate all of the indexes properly to S3/Cloudfiles? Or do they only add
the items when they're indexed? I have read a ton about people trying to do
that, looks like they need to re-index so that the offsite gateways get
populated - is that correct?
The clustering seems to be pretty magical as far as discovery goes - this
is the config that worked great with Rackspace Cloud boxes: gist:1390228 · GitHub
Do the boxes have to be on the same subnet to find each other?
I couldn't find any details about that and it all worked flawlessly, but
wondering how I'll add boxes in the future when they run out of IP
addresses that are "nearby". Or if that even matters.
Sorry if I've missed something, I have read everything I can find and I'm
just trying to make sure on my last few questions. I found a lot of
information that appeared to apply to old releases and just wanted to
clarify.
We're really loving the product and I spent much of this weekend adding
and removing nodes to my cluster, dropping and adding all sorts of indexes
and watching it rebalance - very nice work so far.
On Thu, Nov 24, 2011 at 7:02 AM, Shay Banon kimchy@gmail.com wrote:
To be honest, I am lost, here are some points:
You say you use Rackspace, and it works, but the config points to AWS
configuration?
Sorry for being confusing - we're using a Rackspace cloud VPS with the
S3 gateway.
There is no rackspace cloudfiles support to act as gateway, only s3.
No problem - I thought there was - was testing with S3 and will stick with that.
I must have gotten confused with the Gateway information here:
In any case for 2, you should start with local gateway, its perfectly
fine to use on one node, to many. I don't understand why you used fs gateway
in your one node scenario now.
I figured local should have been used - I didn't actually set that box
up but am stepping in and setting up the cluster now.
Its an "old" post, before local gateway was implemented...
In any case for 2, you should start with local gateway, its perfectly
fine to use on one node, to many. I don't understand why you used fs
gateway
in your one node scenario now.
I figured local should have been used - I didn't actually set that box
up but am stepping in and setting up the cluster now.
Its an "old" post, before local gateway was implemented...
In any case for 2, you should start with local gateway, its perfectly
fine to use on one node, to many. I don't understand why you used fs
gateway
in your one node scenario now.
I figured local should have been used - I didn't actually set that box
up but am stepping in and setting up the cluster now.
Its an "old" post, before local gateway was implemented...
In any case for 2, you should start with local gateway, its
perfectly
fine to use on one node, to many. I don't understand why you used fs
gateway
in your one node scenario now.
I figured local should have been used - I didn't actually set that box
up but am stepping in and setting up the cluster now.
Its an "old" post, before local gateway was implemented...
In any case for 2, you should start with local gateway, its
perfectly
fine to use on one node, to many. I don't understand why you used fs
gateway
in your one node scenario now.
I figured local should have been used - I didn't actually set that box
up but am stepping in and setting up the cluster now.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
