# New feature - new cluster? \[Best Practice\] **URL:** \ **Category:** Elasticsearch\ **Created:** [March 22, 2015, 8:34pm UTC](https://discuss.elastic.co/t/new-feature-new-cluster-best-practice/22803 "2015-03-22T20:34:03Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Nick\_Malcolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nick_malcolm/32/44776_2.png) [@Nick\_Malcolm](https://discuss.elastic.co/u/Nick_Malcolm)\ **Post date:** [March 22, 2015, 8:34pm UTC](https://discuss.elastic.co/t/new-feature-new-cluster-best-practice/22803/1 "2015-03-22T20:34:03Z")
Hi, We have a cluster set up already for indexing customer documents (bigger documents, medium traffic). We want to add event logging (small documents, lots of traffic). _Should we do this on a new cluster?_ - When do you typically create a separate cluster vs. add indexes to a cluster? - Is it straightforward to move indices to a new cluster at a later date? Reasons I can think of for using the same cluster: - Fewer servers to administer - ES is already well equipped to handle multiple indices with various mappings - Simple searching across event & document indices, if needed - If we were planning to add new servers for an events cluster, why not have them in the same cluster for more redundancy Reasons I can think of for a new, separate, cluster: - The cluster can be tuned to the relevant performance needs - Don't have to worry about the flood of events using up all the disk space - Separation of responsibility / "single responsibility" design pattern - If one cluster goes down / hits performance problems, the other can continue along fine I couldn't see any blog posts or documentation around best practice for this, so your insight would be most welcome! Cheers, Nick -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 22, 2015, 10:25pm UTC](https://discuss.elastic.co/t/new-feature-new-cluster-best-practice/22803/2 "2015-03-22T22:25:49Z")
There's no real best practice here at this stage. But if you think about traditional datastores (ie DBs), would you mix these data sets? On 22 March 2015 at 13:34, Nick Malcolm [nick@revert.io](mailto:nick@revert.io) wrote: > Hi, > > We have a cluster set up already for indexing customer documents (bigger > documents, medium traffic). We want to add event logging (small documents, > lots of traffic). > _Should we do this on a new cluster?_ > > - When do you typically create a separate cluster vs. add indexes to a > cluster? > - Is it straightforward to move indices to a new cluster at a later > date? > > Reasons I can think of for using the same cluster: > > - Fewer servers to administer > - ES is already well equipped to handle multiple indices with various > mappings > - Simple searching across event & document indices, if needed > - If we were planning to add new servers for an events cluster, why > not have them in the same cluster for more redundancy > > Reasons I can think of for a new, separate, cluster: > > - The cluster can be tuned to the relevant performance needs > - Don't have to worry about the flood of events using up all the > disk space > > - Separation of responsibility / "single responsibility" design pattern > - If one cluster goes down / hits performance problems, the other can > continue along fine > > I couldn't see any blog posts or documentation around best practice for > this, so your insight would be most welcome! > > Cheers, > Nick > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8gwqsNKvKnUirEBabwiGwfZ3Q6HsikFJQWg0wEL%3DJ2fA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8gwqsNKvKnUirEBabwiGwfZ3Q6HsikFJQWg0wEL%3DJ2fA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Nick\_Malcolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nick_malcolm/32/44776_2.png) [@Nick\_Malcolm](https://discuss.elastic.co/u/Nick_Malcolm)\ **Post date:** [March 22, 2015, 11:35pm UTC](https://discuss.elastic.co/t/new-feature-new-cluster-best-practice/22803/3 "2015-03-22T23:35:44Z")
Hi Mark! Good question - that gave me something to Google. Seems the issue is pretty subjective. I guess the question becomes "Is an index equivalent to a traditional database, or is a cluster equivalent"? In my mind index = traditional database. Types = tables, etc. In which case we would add event indices to the same cluster. Maybe we'll do it that way, and if it hurts later we'll look at putting it on its own cluster, or other performance improvements. On Monday, March 23, 2015 at 11:26:18 AM UTC+13, Mark Walkom wrote: > There's no real best practice here at this stage. > But if you think about traditional datastores (ie DBs), would you mix > these data sets? > > On 22 March 2015 at 13:34, Nick Malcolm \<[ni...@revert.io](mailto:ni...@revert.io) \\> > wrote: > > > Hi, > > > > We have a cluster set up already for indexing customer documents (bigger > > documents, medium traffic). We want to add event logging (small documents, > > lots of traffic). > > _Should we do this on a new cluster?_ > > > > - When do you typically create a separate cluster vs. add indexes to > > a cluster? > > - Is it straightforward to move indices to a new cluster at a later > > date? > > > > Reasons I can think of for using the same cluster: > > > > - Fewer servers to administer > > - ES is already well equipped to handle multiple indices with various > > mappings > > - Simple searching across event & document indices, if needed > > - If we were planning to add new servers for an events cluster, why > > not have them in the same cluster for more redundancy > > > > Reasons I can think of for a new, separate, cluster: > > > > - The cluster can be tuned to the relevant performance needs > > - Don't have to worry about the flood of events using up all the > > disk space > > > > - Separation of responsibility / "single responsibility" design > > pattern > > - If one cluster goes down / hits performance problems, the other can > > continue along fine > > > > I couldn't see any blog posts or documentation around best practice for > > this, so your insight would be most welcome! > > > > Cheers, > > Nick > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/bae2e441-7a7e-4f56-be21-a584ebde5df5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/bae2e441-7a7e-4f56-be21-a584ebde5df5%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 23, 2015, 12:02am UTC](https://discuss.elastic.co/t/new-feature-new-cluster-best-practice/22803/4 "2015-03-23T00:02:49Z")
Seems sane. A lot of these questions are answered based around _your_ use case. If you have low volume for one new dataset, it totally makes sense to leverage existing infrastructure and then make future decisions as you go. On 23 March 2015 at 10:35, Nick Malcolm [nick@revert.io](mailto:nick@revert.io) wrote: > Hi Mark! > > Good question - that gave me something to Google. Seems the issue is > pretty subjective. > > I guess the question becomes "Is an index equivalent to a traditional > database, or is a cluster equivalent"? In my mind index = traditional > database. Types = tables, etc. In which case we would add event indices to > the same cluster. > > Maybe we'll do it that way, and if it hurts later we'll look at putting it > on its own cluster, or other performance improvements. > > On Monday, March 23, 2015 at 11:26:18 AM UTC+13, Mark Walkom wrote: > > > There's no real best practice here at this stage. > > But if you think about traditional datastores (ie DBs), would you mix > > these data sets? > > > > On 22 March 2015 at 13:34, Nick Malcolm [ni...@revert.io](mailto:ni...@revert.io) wrote: > > > > > Hi, > > > > > > We have a cluster set up already for indexing customer documents (bigger > > > documents, medium traffic). We want to add event logging (small documents, > > > lots of traffic). > > > _Should we do this on a new cluster?_ > > > > > > - When do you typically create a separate cluster vs. add indexes to > > > a cluster? > > > - Is it straightforward to move indices to a new cluster at a later > > > date? > > > > > > Reasons I can think of for using the same cluster: > > > > > > - Fewer servers to administer > > > - ES is already well equipped to handle multiple indices with > > > various mappings > > > - Simple searching across event & document indices, if needed > > > - If we were planning to add new servers for an events cluster, why > > > not have them in the same cluster for more redundancy > > > > > > Reasons I can think of for a new, separate, cluster: > > > > > > - The cluster can be tuned to the relevant performance needs > > > - Don't have to worry about the flood of events using up all the > > > disk space > > > > > > - Separation of responsibility / "single responsibility" design > > > pattern > > > - If one cluster goes down / hits performance problems, the other > > > can continue along fine > > > > > > I couldn't see any blog posts or documentation around best practice for > > > this, so your insight would be most welcome! > > > > > > Cheers, > > > Nick > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b% > > > [40googlegroups.com](http://40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/ebb7afb0-9efb-45b1-b6e7-dba67bb52d1b%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/bae2e441-7a7e-4f56-be21-a584ebde5df5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/bae2e441-7a7e-4f56-be21-a584ebde5df5%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/bae2e441-7a7e-4f56-be21-a584ebde5df5%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/bae2e441-7a7e-4f56-be21-a584ebde5df5%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-yQ2C0PG2M1mHuF7%3D1bMBhtPwBAxywM6qHh4bvETfMJA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-yQ2C0PG2M1mHuF7%3D1bMBhtPwBAxywM6qHh4bvETfMJA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:25am UTC](https://discuss.elastic.co/t/new-feature-new-cluster-best-practice/22803/5 "2017-07-06T00:25:05Z")