I have several large indices (100M docs) on the same cluster. Is there any
advice of when it is appropriate to separate into multiple clusters vs one
large one? Each index has a slightly different usage profile (read vs write
heavy, update vs insert). How many indices would you recommend for a single
cluster? Is it ok to have many large indices on the same cluster?
Depends what your hardware profiles are like, and a bunch of other things
related to you and your environment.
eg If you have high end servers then it makes sense to put your heavy
read/write indexes into a cluster on those, then leave the rest for more
average machines.
We have multiple clusters based on use. One for an application text based
search, one for application logging, one for system logging and we're going
to spin up another one for a new project we're starting. This might sound
like a waste of resources, and it probably is to a degree, but we have the
infrastructure for it and it makes things easier to manage.
I have several large indices (100M docs) on the same cluster. Is there any
advice of when it is appropriate to separate into multiple clusters vs one
large one? Each index has a slightly different usage profile (read vs write
heavy, update vs insert). How many indices would you recommend for a single
cluster? Is it ok to have many large indices on the same cluster?
Thanks Mark! We're deploying on EC2 (always a good time). Seems like the
mixture of different indices that have different usage profiles is leading
to some performance issues that a dedicated cluster would be more
appropriate for.
On Wednesday, July 23, 2014 7:04:34 PM UTC-4, Mark Walkom wrote:
Depends what your hardware profiles are like, and a bunch of other things
related to you and your environment.
eg If you have high end servers then it makes sense to put your heavy
read/write indexes into a cluster on those, then leave the rest for more
average machines.
We have multiple clusters based on use. One for an application text based
search, one for application logging, one for system logging and we're going
to spin up another one for a new project we're starting. This might sound
like a waste of resources, and it probably is to a degree, but we have the
infrastructure for it and it makes things easier to manage.
On 24 July 2014 00:34, Alex Kehayias <al...@shareablee.com <javascript:>>
wrote:
I have several large indices (100M docs) on the same cluster. Is there
any advice of when it is appropriate to separate into multiple clusters vs
one large one? Each index has a slightly different usage profile (read vs
write heavy, update vs insert). How many indices would you recommend for a
single cluster? Is it ok to have many large indices on the same cluster?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.