Clustering ElasticSearch in "local" mode


(Michael Smolyak) #1

ElasticSearch Java API documentation describes the "local" mode
of operation (
http://www.elasticsearch.org/guide/reference/java-api/client.html) where
the ES server runs in the same JVM as the parent process. We are using this
mode with our Grails application (through the ElasticSearch Grails Plugin).
It works fine in case of a single Tomcat server running the Grails
application. I am trying to cluster the Grails app by using a cluster of
Tomcat servers. I wonder whether it is possible to cluster ElasticSearch
servers running in the local mode (i.e., each runs in the same JVM as its
parent Tomcat process). Is it possible to tell all the "local" ES servers
to join the same cluster (and if so, how) or do I have to use the "node"
mode by running ElasticSearch as a separate server process to achieve
clustering?

Thank you,

Michael

--


(Lukáš Vlček) #2

Hi,
If you are saying that each ES node runs in different JVM then I don't
think it is possible.
Lukas
Dne 26.9.2012 15:14 "Michael Smolyak" msmolyak.nextcentury@gmail.com
napsal(a):

ElasticSearch Java API documentation describes the "local" mode
of operation (
http://www.elasticsearch.org/guide/reference/java-api/client.html) where
the ES server runs in the same JVM as the parent process. We are using this
mode with our Grails application (through the ElasticSearch Grails Plugin).
It works fine in case of a single Tomcat server running the Grails
application. I am trying to cluster the Grails app by using a cluster of
Tomcat servers. I wonder whether it is possible to cluster ElasticSearch
servers running in the local mode (i.e., each runs in the same JVM as its
parent Tomcat process). Is it possible to tell all the "local" ES servers
to join the same cluster (and if so, how) or do I have to use the "node"
mode by running ElasticSearch as a separate server process to achieve
clustering?

Thank you,

Michael

--

--


(David Pilato) #3

If I understood your use case, I think that you could start a default node
(without local(true)) and use then autodiscovery (by default) to build a cluster
of ES nodes embedded in your tomcat webapp instances.

Why do you think you have to set your node as local?

David.

Le 26 septembre 2012 à 15:43, "Lukáš Vl?ek" lukas.vlcek@gmail.com a écrit :

Hi,
If you are saying that each ES node runs in different JVM then I don't think
it is possible.
Lukas

Dne 26.9.2012 15:14 "Michael Smolyak" < msmolyak.nextcentury@gmail.com
mailto:msmolyak.nextcentury@gmail.com > napsal(a):

ElasticSearch Java API documentation describes the "local" mode of
operation (
http://www.elasticsearch.org/guide/reference/java-api/client.html
http://www.elasticsearch.org/guide/reference/java-api/client.html )
where the ES server runs in the same JVM as the parent process. We are
using this mode with our Grails application (through the ElasticSearch
Grails Plugin). It works fine in case of a single Tomcat server running
the Grails application. I am trying to cluster the Grails app by using
a cluster of Tomcat servers. I wonder whether it is possible to cluster
ElasticSearch servers running in the local mode (i.e., each runs in the
same JVM as its parent Tomcat process). Is it possible to tell all the
"local" ES servers to join the same cluster (and if so, how) or do I
have to use the "node" mode by running ElasticSearch as a separate
server process to achieve clustering?

Thank you,

Michael

--

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--


(Michael Smolyak) #4

David,

Thank you for your response. What I have is a Grails application running in
Tomcat. Grails ElasticSearch plugin supports three modes (based on ES Java
API): local, node and transport. In local mode the ES server is started
implicitly, while in node mode, (my understanding) I would have to start ES
server as a separate process, which would complicate the overall setup.
Thus I would like to run an embedded ES server, but be able to cluster them.

When you mention starting the default node, does it have to be started as a
separate process? What do you mean by embedded then? How is embedded
"local" different mean from embedded "node"?

Michael

On Wednesday, September 26, 2012 9:49:34 AM UTC-4, David Pilato wrote:

If I understood your use case, I think that you could start a default
node (without local(true)) and use then autodiscovery (by default) to build
a cluster of ES nodes embedded in your tomcat webapp instances.

Why do you think you have to set your node as local?

David.

Le 26 septembre 2012 à 15:43, "Lukáš Vl?ek" <lukas...@gmail.com<javascript:>>
a écrit :

Hi,
If you are saying that each ES node runs in different JVM then I don't
think it is possible.
Lukas
Dne 26.9.2012 15:14 "Michael Smolyak" < msmolyak.n...@gmail.com<javascript:>>
napsal(a):

ElasticSearch Java API documentation describes the "local" mode
of operation (
http://www.elasticsearch.org/guide/reference/java-api/client.html) where
the ES server runs in the same JVM as the parent process. We are using this
mode with our Grails application (through the ElasticSearch Grails Plugin).
It works fine in case of a single Tomcat server running the Grails
application. I am trying to cluster the Grails app by using a cluster of
Tomcat servers. I wonder whether it is possible to cluster ElasticSearch
servers running in the local mode (i.e., each runs in the same JVM as its
parent Tomcat process). Is it possible to tell all the "local" ES servers
to join the same cluster (and if so, how) or do I have to use the "node"
mode by running ElasticSearch as a separate server process to achieve
clustering?

Thank you,

Michael

--

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--


(Michael Smolyak) #5

The solution was to use the "data node" by using (Java API)

nodeBuilder.client(false)
nodeBuilder.data(true)

In that mode I do not need to run a separate server outside of the Web
application and the two data nodes in Tomcat cluster can see each other.

On Wednesday, September 26, 2012 10:16:20 AM UTC-4, Michael Smolyak wrote:

David,

Thank you for your response. What I have is a Grails application running
in Tomcat. Grails ElasticSearch plugin supports three modes (based on ES
Java API): local, node and transport. In local mode the ES server is
started implicitly, while in node mode, (my understanding) I would have to
start ES server as a separate process, which would complicate the overall
setup. Thus I would like to run an embedded ES server, but be able to
cluster them.

When you mention starting the default node, does it have to be started as
a separate process? What do you mean by embedded then? How is embedded
"local" different mean from embedded "node"?

Michael

On Wednesday, September 26, 2012 9:49:34 AM UTC-4, David Pilato wrote:

If I understood your use case, I think that you could start a default
node (without local(true)) and use then autodiscovery (by default) to build
a cluster of ES nodes embedded in your tomcat webapp instances.

Why do you think you have to set your node as local?

David.

Le 26 septembre 2012 à 15:43, "Lukáš Vl?ek" lukas...@gmail.com a
écrit :

Hi,
If you are saying that each ES node runs in different JVM then I don't
think it is possible.
Lukas
Dne 26.9.2012 15:14 "Michael Smolyak" < msmolyak.n...@gmail.com>
napsal(a):

ElasticSearch Java API documentation describes the "local" mode
of operation (
http://www.elasticsearch.org/guide/reference/java-api/client.html) where
the ES server runs in the same JVM as the parent process. We are using this
mode with our Grails application (through the ElasticSearch Grails Plugin).
It works fine in case of a single Tomcat server running the Grails
application. I am trying to cluster the Grails app by using a cluster of
Tomcat servers. I wonder whether it is possible to cluster ElasticSearch
servers running in the local mode (i.e., each runs in the same JVM as its
parent Tomcat process). Is it possible to tell all the "local" ES servers
to join the same cluster (and if so, how) or do I have to use the "node"
mode by running ElasticSearch as a separate server process to achieve
clustering?

Thank you,

Michael

--

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--


(David Pilato) #6

IMHO, that's the default values when you create a node with the Java API.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 sept. 2012 à 23:19, Michael Smolyak msmolyak.nextcentury@gmail.com a écrit :

The solution was to use the "data node" by using (Java API)

			nodeBuilder.client(false)
			nodeBuilder.data(true)

In that mode I do not need to run a separate server outside of the Web application and the two data nodes in Tomcat cluster can see each other.

On Wednesday, September 26, 2012 10:16:20 AM UTC-4, Michael Smolyak wrote:

David,

Thank you for your response. What I have is a Grails application running in Tomcat. Grails ElasticSearch plugin supports three modes (based on ES Java API): local, node and transport. In local mode the ES server is started implicitly, while in node mode, (my understanding) I would have to start ES server as a separate process, which would complicate the overall setup. Thus I would like to run an embedded ES server, but be able to cluster them.

When you mention starting the default node, does it have to be started as a separate process? What do you mean by embedded then? How is embedded "local" different mean from embedded "node"?

Michael

On Wednesday, September 26, 2012 9:49:34 AM UTC-4, David Pilato wrote:

If I understood your use case, I think that you could start a default node (without local(true)) and use then autodiscovery (by default) to build a cluster of ES nodes embedded in your tomcat webapp instances.

Why do you think you have to set your node as local?

David.

Le 26 septembre 2012 à 15:43, "Lukáš Vl?ek" lukas...@gmail.com a écrit :

Hi,
If you are saying that each ES node runs in different JVM then I don't think it is possible.
Lukas

Dne 26.9.2012 15:14 "Michael Smolyak" < msmolyak.n...@gmail.com> napsal(a):
ElasticSearch Java API documentation describes the "local" mode of operation ( http://www.elasticsearch.org/guide/reference/java-api/client.html) where the ES server runs in the same JVM as the parent process. We are using this mode with our Grails application (through the ElasticSearch Grails Plugin). It works fine in case of a single Tomcat server running the Grails application. I am trying to cluster the Grails app by using a cluster of Tomcat servers. I wonder whether it is possible to cluster ElasticSearch servers running in the local mode (i.e., each runs in the same JVM as its parent Tomcat process). Is it possible to tell all the "local" ES servers to join the same cluster (and if so, how) or do I have to use the "node" mode by running ElasticSearch as a separate server process to achieve clustering?

Thank you,

Michael

--

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

--


(system) #7