We are using ES for all our reads, including resolving relations
between entities. Updates go into MySQL first and if the transaction
commits, we replicate the data to ES.
Assuming 90% reads and 10% writes, our app is mainly ES bound. We have
chosen to run ES embedded instead of using a standalone ES cluster.
Each of our app nodes is also an ES node. This makes it that our unit
testing setup is exactly the same as our production setup. Both use
embedded ES.
Is there something we should know that could make running these
embedded ES less production ready or less scalable than running a
dedicated cluster?
IMHO it is better to run ES as a standalone process. I am not 100% sure from
the top of my head but running embedded ES means that it shares the same JVM
with other processes (like Tomcat for example if you embed ES into web app)
which makes it more difficult to fine tune. Also bringing down your
application brings down ES too (is that what you really want, does upgrade
to your app means that ES has to go down too? And if you want to upgrade to
a new version of ES in the future does it mean you also want to shutdown
your app?).
Regards,
Lukas
On Mon, Jun 27, 2011 at 2:22 AM, Remy Gendron remy@arrova.ca wrote:
Hi all,
We are using ES for all our reads, including resolving relations
between entities. Updates go into MySQL first and if the transaction
commits, we replicate the data to ES.
Assuming 90% reads and 10% writes, our app is mainly ES bound. We have
chosen to run ES embedded instead of using a standalone ES cluster.
Each of our app nodes is also an ES node. This makes it that our unit
testing setup is exactly the same as our production setup. Both use
embedded ES.
Is there something we should know that could make running these
embedded ES less production ready or less scalable than running a
dedicated cluster?
As long as they form the same cluster, then there isn't a difference compared to running a dedicated elasticsearch cluster. But, as Lukas noted, you now have more moving parts when it comes to fine tuning the JVM and understanding what can lead to problems.
On Monday, June 27, 2011 at 11:56 AM, Lukáš Vlček wrote:
Hi,
IMHO it is better to run ES as a standalone process. I am not 100% sure from the top of my head but running embedded ES means that it shares the same JVM with other processes (like Tomcat for example if you embed ES into web app) which makes it more difficult to fine tune. Also bringing down your application brings down ES too (is that what you really want, does upgrade to your app means that ES has to go down too? And if you want to upgrade to a new version of ES in the future does it mean you also want to shutdown your app?).
Regards,
Lukas
On Mon, Jun 27, 2011 at 2:22 AM, Remy Gendron <remy@arrova.ca (mailto:remy@arrova.ca)> wrote:
Hi all,
We are using ES for all our reads, including resolving relations
between entities. Updates go into MySQL first and if the transaction
commits, we replicate the data to ES.
Assuming 90% reads and 10% writes, our app is mainly ES bound. We have
chosen to run ES embedded instead of using a standalone ES cluster.
Each of our app nodes is also an ES node. This makes it that our unit
testing setup is exactly the same as our production setup. Both use
embedded ES.
Is there something we should know that could make running these
embedded ES less production ready or less scalable than running a
dedicated cluster?
Thank you both. As we are mostly in beta stage, we are willing to live
with the process coupling for now in exchange for the reduced
deployment complexity. It's great to know the embedded mode is not
just for unit testing.
Thanks again Shay, we find that ES is such a great asset to our
project.
As long as they form the same cluster, then there isn't a difference compared to running a dedicated elasticsearch cluster. But, as Lukas noted, you now have more moving parts when it comes to fine tuning the JVM and understanding what can lead to problems.
On Monday, June 27, 2011 at 11:56 AM, Lukáš Vlček wrote:
Hi,
IMHO it is better to run ES as a standalone process. I am not 100% sure from the top of my head but running embedded ES means that it shares the same JVM with other processes (like Tomcat for example if you embed ES into web app) which makes it more difficult to fine tune. Also bringing down your application brings down ES too (is that what you really want, does upgrade to your app means that ES has to go down too? And if you want to upgrade to a new version of ES in the future does it mean you also want to shutdown your app?).
Regards,
Lukas
On Mon, Jun 27, 2011 at 2:22 AM, Remy Gendron <r...@arrova.ca (mailto:r...@arrova.ca)> wrote:
Hi all,
We are using ES for all our reads, including resolving relations
between entities. Updates go into MySQL first and if the transaction
commits, we replicate the data to ES.
Assuming 90% reads and 10% writes, our app is mainly ES bound. We have
chosen to run ES embedded instead of using a standalone ES cluster.
Each of our app nodes is also an ES node. This makes it that our unit
testing setup is exactly the same as our production setup. Both use
embedded ES.
Is there something we should know that could make running these
embedded ES less production ready or less scalable than running a
dedicated cluster?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.