Multiple Elastic Search Instances on one machine


(Christopher Hunter) #1

Hi,

We're just starting to use ElasticSearch in our application, and so far it's
working great, so thanks for all the hard work on it, its really good out of
the box. We had no problem getting the service up and running thanks to the
GitHub documentation.

One question about running multiple instances though. We have a fairly
large VPS that we run two concurrent development tomcat instances, which
functionally are completely isolated from each other. We'd like to run two
separate instances of ElasticSearch as well on the same machine, separate
data/logging/et al. We would also potentially need the ability to run
separate versions per tomcat instance as well. I googled around a bit, and
searched this list and didn't find much on this. Is there a recommended way?

Thanks!

Hunter


(Karussell) #2

ES does not run in tomcat it uses netty. E.g. see:

http://stackoverflow.com/questions/4739308/elasticsearch-solr-on-aws-elastic-beanstalk

I'm not sure about wether you can use two separate instances, but I
think so (I'm also new to ES).

just make sure they are using different ports and data directories
IMHO

On 4 Feb., 04:24, Christopher Hunter hunter.christop...@gmail.com
wrote:

Hi,

We're just starting to use ElasticSearch in our application, and so far it's
working great, so thanks for all the hard work on it, its really good out of
the box. We had no problem getting the service up and running thanks to the
GitHub documentation.

One question about running multiple instances though. We have a fairly
large VPS that we run two concurrent development tomcat instances, which
functionally are completely isolated from each other. We'd like to run two
separate instances of ElasticSearch as well on the same machine, separate
data/logging/et al. We would also potentially need the ability to run
separate versions per tomcat instance as well. I googled around a bit, and
searched this list and didn't find much on this. Is there a recommended way?

Thanks!

Hunter


(Travis Groth) #3
I'm guessing by what you're saying, you want to keep those two
instances from clustering or otherwise interacting with each other? 
I am relatively new to ES, but I think I can point you at the right
info.




Overview of where the config lives/general usage:

http://www.elasticsearch.com/docs/elasticsearch/setup/configuration

Depending on whether you use different addresses (say, loopbacks) or
want to use different port settings to keep them from speaking to
each other, review these pages:

http://www.elasticsearch.com/docs/elasticsearch/modules/discovery

http://www.elasticsearch.com/docs/elasticsearch/modules/node/network/

http://www.elasticsearch.com/docs/elasticsearch/modules/transport/

http://www.elasticsearch.com/docs/elasticsearch/modules/http/

Basically you'd have a directory per instance.  Then you could
either bind them to different addresses entirely, or tell the http
and transport modules to use alternate ports.  In both cases you
should probably set the discovery cluster name differently as an
extra layer of protection.  If you use different IP addresses you
would almost definitely need to do that to keep them from joining
each other.  With alternate ports I'm not entirely sure but I don't
think it would hurt.




Travis
















On 02/03/2011 10:24 PM, Christopher Hunter wrote:
<blockquote cite="mid:AANLkTinQoNYxcymwcVSJ1DRt=oAsc_xvHOJ+HM1FfB9-@mail.gmail.com" type="cite">Hi, 




      We're just starting to use ElasticSearch in our application,
      and so far it's working great, so thanks for all the hard work
      on it, its really good out of the box.  We had no problem
      getting the service up and running thanks to the GitHub
      documentation. 




      One question about running multiple instances though.  We have
      a fairly large VPS that we run two concurrent development
      tomcat instances, which functionally are completely isolated
      from each other.  We'd like to run two separate instances of
      ElasticSearch as well on the same machine, separate
      data/logging/et al.  We would also potentially need the
      ability to run separate versions per tomcat instance as
      well. I googled around a bit, and searched this list and
      didn't find much on this. Is there a recommended way?  

Thanks!

              Hunter

(Shay Banon) #4

The simplest is to use a different cluster name for the instances. You might need to explicitly set ports (transport / http) so you will know which one each different instance is going to use, though ES will automatically bind to the next port if the first one is taken.
On Saturday, February 5, 2011 at 6:40 AM, Travis Groth wrote:

I'm guessing by what you're saying, you want to keep those two instances from clustering or otherwise interacting with each other? I am relatively new to ES, but I think I can point you at the right info.

Overview of where the config lives/general usage:

http://www.elasticsearch.com/docs/elasticsearch/setup/configuration

Depending on whether you use different addresses (say, loopbacks) or want to use different port settings to keep them from speaking to each other, review these pages:

http://www.elasticsearch.com/docs/elasticsearch/modules/discovery

http://www.elasticsearch.com/docs/elasticsearch/modules/node/network/

http://www.elasticsearch.com/docs/elasticsearch/modules/transport/

http://www.elasticsearch.com/docs/elasticsearch/modules/http/

Basically you'd have a directory per instance. Then you could either bind them to different addresses entirely, or tell the http and transport modules to use alternate ports. In both cases you should probably set the discovery cluster name differently as an extra layer of protection. If you use different IP addresses you would almost definitely need to do that to keep them from joining each other. With alternate ports I'm not entirely sure but I don't think it would hurt.

Travis

On 02/03/2011 10:24 PM, Christopher Hunter wrote:

Hi,

We're just starting to use ElasticSearch in our application, and so far it's working great, so thanks for all the hard work on it, its really good out of the box. We had no problem getting the service up and running thanks to the GitHub documentation.

One question about running multiple instances though. We have a fairly large VPS that we run two concurrent development tomcat instances, which functionally are completely isolated from each other. We'd like to run two separate instances of ElasticSearch as well on the same machine, separate data/logging/et al. We would also potentially need the ability to run separate versions per tomcat instance as well. I googled around a bit, and searched this list and didn't find much on this. Is there a recommended way?

Thanks!

Hunter


(system) #5