# Separating Index and Search **URL:** \ **Category:** Elasticsearch\ **Created:** [February 19, 2013, 12:37pm UTC](https://discuss.elastic.co/t/separating-index-and-search/10798 "2013-02-19T12:37:26Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Ophir\_Michaeli](https://avatars.discourse-cdn.com/v4/letter/o/9f8e36/32.png) [@Ophir\_Michaeli](https://discuss.elastic.co/u/Ophir_Michaeli)\ **Post date:** [February 19, 2013, 12:37pm UTC](https://discuss.elastic.co/t/separating-index-and-search/10798/1 "2013-02-19T12:37:26Z")
Hi, My question is about the best practice to divide elasticsearch indexing and search systems. Currently, without elasticsearch (and with lucene), our indexing works on several machines at one location and the search machines are at another location. The indexed data is copied or updated periodically from the indexing machines to the search machines. We want to maintain a similar structure using elasticsearch. Is it possible for the elasticsearch nodes on the indexing and search machines to be on the same cluster, so the indexing nodes will put replicas on the search nodes and will keep the replicas updated, while the search client will approach only the search nodes (ignoring the indexing nodes because they are in a different location and approaching them will slow the search). Also – is it possible to set the indexing nodes on the cluster to update the search nodes periodically (and not constantly so the search performance won't decrease)? Thanks, Ophir -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\ **Post date:** [February 19, 2013, 1:43pm UTC](https://discuss.elastic.co/t/separating-index-and-search/10798/2 "2013-02-19T13:43:43Z")
I'm not entirely positive, so wait for someone with more experience to confirm/deny...but I don't think this is quite possible in ES right now. You can fake it with, shard allocation filtering[http://www.elasticsearch.org/guide/reference/modules/cluster.html](http://www.elasticsearch.org/guide/reference/modules/cluster.html), multiple indices and aliases, however. First, let's talk about the solution that _appears_ to work, but in fact does not: forced awareness settings. Forced awareness basically prevents duplication of data within the same zone, so a primary + replica cannot live in the same zone. Imagine you have two nodes in your "indexing" zone, and two nodes in your "search" zone. Primary shards are allocated in "indexing", replicas on "search". If you use forced awareness and a node in your "search" zone goes down, ES will know avoid initializing a corresponding replica in your indexing zone, since the primary already lives there. Even better, if you perform searches on the "search" zone, forced awareness makes ES prefer querying nodes in the same zone. Great! However, the problem arises if one of your indexing nodes goes down. Zones enforce data duplication boundaries, but does not interfere with primary promotion. If one of your indexing nodes goes down, your cluster is now missing a primary shard. ES has no choice but to promote a replica to a primary, even if it lives in another zone. Now your indexing node is actually living in the "search" zone and everything is all messed up. As an alternative, what you can do is use Shard Allocation Filtering to separate an "Indexing" index and a "Search" index onto physically separate nodes. E.g. search nodes are forced to allocate to nodes with the "search" tag. You then index into your "indexing" index (hah) and when it is ready for search requests, change the tags on the index over to "search". ES will automatically transfer the shards over to your Search nodes. When the transfer is complete, change a top-level alias to switch between the old and new index transparently, then delete the old index. This method obviously has a lot of moving parts, and loads the search nodes with periodic network transfer as you move shards around. -Zach On Tuesday, February 19, 2013 7:37:26 AM UTC-5, Ophir Michaeli wrote: > Hi, > > My question is about the best practice to divide elasticsearch indexing > and search systems. > > Currently, without elasticsearch (and with lucene), our indexing works on > several machines at one location and the search machines are at another > location. > > The indexed data is copied or updated periodically from the indexing > machines to the search machines. > > We want to maintain a similar structure using elasticsearch. > > Is it possible for the elasticsearch nodes on the indexing and search > machines to be on the same cluster, > > so the indexing nodes will put replicas on the search nodes and will keep > the replicas updated, while the search client will approach only the search > nodes (ignoring the indexing nodes because they are in a different location > and approaching them will slow the search). > > Also – is it possible to set the indexing nodes on the cluster to update > the search nodes periodically (and not constantly so the search performance > won't decrease)? > > Thanks, > > Ophir -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ophir\_Michaeli](https://avatars.discourse-cdn.com/v4/letter/o/9f8e36/32.png) [@Ophir\_Michaeli](https://discuss.elastic.co/u/Ophir_Michaeli)\ **Post date:** [February 19, 2013, 4:40pm UTC](https://discuss.elastic.co/t/separating-index-and-search/10798/3 "2013-02-19T16:40:50Z")
Thanks for the explanation. Regarding the second suggestion, is it the same as setting an "Indexing" cluster, and a "Search" cluster? The index created by the Indexing cluster periodically copied to the Search cluster folder (index) to replace the old search folder (index). Both this and your suggested solution has the disadvantage of not allowing real time indexed data to be searchable, which is one of our considerations to move to elasticsearch. On Tuesday, February 19, 2013 2:37:26 PM UTC+2, Ophir Michaeli wrote: > Hi, > > My question is about the best practice to divide elasticsearch indexing > and search systems. > > Currently, without elasticsearch (and with lucene), our indexing works on > several machines at one location and the search machines are at another > location. > > The indexed data is copied or updated periodically from the indexing > machines to the search machines. > > We want to maintain a similar structure using elasticsearch. > > Is it possible for the elasticsearch nodes on the indexing and search > machines to be on the same cluster, > > so the indexing nodes will put replicas on the search nodes and will keep > the replicas updated, while the search client will approach only the search > nodes (ignoring the indexing nodes because they are in a different location > and approaching them will slow the search). > > Also – is it possible to set the indexing nodes on the cluster to update > the search nodes periodically (and not constantly so the search performance > won't decrease)? > > Thanks, > > Ophir -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [February 19, 2013, 5:53pm UTC](https://discuss.elastic.co/t/separating-index-and-search/10798/4 "2013-02-19T17:53:16Z")
The infrastructure that we had in place before moving to Elasticsearch had the same workflow. Indexer nodes create the index which then gets deployed to searcher nodes. For probably the same performance reasons, we tried to replicate the same workflow in Elasticsearch, using many of the techniques that Zachary highlighted. In the end we found it to be too much of an administrative hassle. We decided to embrace Elasticsearch to its fullest and let it deal with the merging of new docs/segments in an efficient manner. If you want real time indexing, you need to have the searcher nodes handle the indexing. Aliases are a good way to handle the creation of new indices that are not meant to be searched yet. Cheers, Ivan On Tue, Feb 19, 2013 at 8:40 AM, Ophir Michaeli [ophirmichaeli@gmail.com](mailto:ophirmichaeli@gmail.com)wrote: > Thanks for the explanation.\*\*\*\*\*\* > > Regarding the second suggestion, is it the same as setting an "Indexing" > cluster, and a "Search" cluster?\*\*\*\* > > The index created by the Indexing cluster periodically copied to the > Search cluster folder (index) to replace the old search folder (index).\*\*\* > \* > > Both this and your suggested solution has the disadvantage of not allowing > real time indexed data to be searchable, which is one of our considerations > to move to elasticsearch.\*\*\*\* > > On Tuesday, February 19, 2013 2:37:26 PM UTC+2, Ophir Michaeli wrote: > > > Hi, > > > > My question is about the best practice to divide elasticsearch indexing > > and search systems. > > > > Currently, without elasticsearch (and with lucene), our indexing works on > > several machines at one location and the search machines are at another > > location. > > > > The indexed data is copied or updated periodically from the indexing > > machines to the search machines. > > > > We want to maintain a similar structure using elasticsearch. > > > > Is it possible for the elasticsearch nodes on the indexing and search > > machines to be on the same cluster, > > > > so the indexing nodes will put replicas on the search nodes and will keep > > the replicas updated, while the search client will approach only the search > > nodes (ignoring the indexing nodes because they are in a different location > > and approaching them will slow the search). > > > > Also – is it possible to set the indexing nodes on the cluster to update > > the search nodes periodically (and not constantly so the search performance > > won't decrease)? > > > > Thanks, > > > > Ophir > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\ **Post date:** [February 19, 2013, 6:30pm UTC](https://discuss.elastic.co/t/separating-index-and-search/10798/5 "2013-02-19T18:30:04Z")
It is similar to having two separate clusters, except the method I outlined lets Elasticsearch deal with data migration so you don't have to rsync folders around manually. Otherwise it is basically the same. And basically everything Ivan said =) -Zach On Tuesday, February 19, 2013 11:40:50 AM UTC-5, Ophir Michaeli wrote: > Thanks for the explanation. > > Regarding the second suggestion, is it the same as setting an "Indexing" > cluster, and a "Search" cluster? > > The index created by the Indexing cluster periodically copied to the > Search cluster folder (index) to replace the old search folder (index). > > Both this and your suggested solution has the disadvantage of not allowing > real time indexed data to be searchable, which is one of our considerations > to move to elasticsearch. > > On Tuesday, February 19, 2013 2:37:26 PM UTC+2, Ophir Michaeli wrote: > > > Hi, > > > > My question is about the best practice to divide elasticsearch indexing > > and search systems. > > > > Currently, without elasticsearch (and with lucene), our indexing works on > > several machines at one location and the search machines are at another > > location. > > > > The indexed data is copied or updated periodically from the indexing > > machines to the search machines. > > > > We want to maintain a similar structure using elasticsearch. > > > > Is it possible for the elasticsearch nodes on the indexing and search > > machines to be on the same cluster, > > > > so the indexing nodes will put replicas on the search nodes and will keep > > the replicas updated, while the search client will approach only the search > > nodes (ignoring the indexing nodes because they are in a different location > > and approaching them will slow the search). > > > > Also – is it possible to set the indexing nodes on the cluster to update > > the search nodes periodically (and not constantly so the search performance > > won't decrease)? > > > > Thanks, > > > > Ophir -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ophir\_Michaeli](https://avatars.discourse-cdn.com/v4/letter/o/9f8e36/32.png) [@Ophir\_Michaeli](https://discuss.elastic.co/u/Ophir_Michaeli)\ **Post date:** [February 23, 2013, 3:16pm UTC](https://discuss.elastic.co/t/separating-index-and-search/10798/6 "2013-02-23T15:16:51Z")
Zachary – Suppose I want to use suggestion 1 – I managed to set zones, it's not clear how to check that it works, meaning: 1. Primary shards are at indexing zone and replica shards are at search zone. 2. Search is done only at search zone. Thanks, Ophir On Tuesday, February 19, 2013 8:30:04 PM UTC+2, Zachary Tong wrote: > It is similar to having two separate clusters, except the method I > outlined lets Elasticsearch deal with data migration so you don't have to > rsync folders around manually. Otherwise it is basically the same. > > And basically everything Ivan said =) > > -Zach > > On Tuesday, February 19, 2013 11:40:50 AM UTC-5, Ophir Michaeli wrote: > > > Thanks for the explanation. > > > > Regarding the second suggestion, is it the same as setting an "Indexing" > > cluster, and a "Search" cluster? > > > > The index created by the Indexing cluster periodically copied to the > > Search cluster folder (index) to replace the old search folder (index). > > > > Both this and your suggested solution has the disadvantage of not > > allowing real time indexed data to be searchable, which is one of our > > considerations to move to elasticsearch. > > > > On Tuesday, February 19, 2013 2:37:26 PM UTC+2, Ophir Michaeli wrote: > > > > > Hi, > > > > > > My question is about the best practice to divide elasticsearch indexing > > > and search systems. > > > > > > Currently, without elasticsearch (and with lucene), our indexing works > > > on several machines at one location and the search machines are at another > > > location. > > > > > > The indexed data is copied or updated periodically from the indexing > > > machines to the search machines. > > > > > > We want to maintain a similar structure using elasticsearch. > > > > > > Is it possible for the elasticsearch nodes on the indexing and search > > > machines to be on the same cluster, > > > > > > so the indexing nodes will put replicas on the search nodes and will > > > keep the replicas updated, while the search client will approach only the > > > search nodes (ignoring the indexing nodes because they are in a different > > > location and approaching them will slow the search). > > > > > > Also – is it possible to set the indexing nodes on the cluster to update > > > the search nodes periodically (and not constantly so the search performance > > > won't decrease)? > > > > > > Thanks, > > > > > > Ophir -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\ **Post date:** [February 24, 2013, 3:55pm UTC](https://discuss.elastic.co/t/separating-index-and-search/10798/7 "2013-02-24T15:55:26Z")
Hi Ophir, Based on this comment by Shay[https://github.com/elasticsearch/elasticsearch/issues/1352#issuecomment-2166409](https://github.com/elasticsearch/elasticsearch/issues/1352#issuecomment-2166409), it appears that you can check attribute status through the Cluster State API, but cannot check the awareness settings. I think the only way you will be able to confirm zone delineation is to manually check to see if the nodes you expect in the "indexing" zone are holding all primary shards. And of course, if (when) an indexing node goes down, you'll have to manually intervene to reroute primaries back out of the "search" zone. Search requests going to a particular zone are "preferentially" served by nodes in the same zone. I'm unsure the degree of this "preference", and if it will ever spill over to the other zone. Someone more experienced with zones would have to answer that. You may be able to manually confirm that search requests are being served by one zone through logging. But, as mentioned before, this use-case is largely tangential to the Elasticsearch "philosophy" and zones were not really designed for this kind of work. On Saturday, February 23, 2013 10:16:51 AM UTC-5, Ophir Michaeli wrote: > Zachary – > > Suppose I want to use suggestion 1 – > I managed to set zones, it's not clear how to check that it works, > meaning: > > 1. Primary shards are at indexing zone and replica shards are at search > zone. > 2. Search is done only at search zone. > > Thanks, Ophir > > On Tuesday, February 19, 2013 8:30:04 PM UTC+2, Zachary Tong wrote: > > > It is similar to having two separate clusters, except the method I > > outlined lets Elasticsearch deal with data migration so you don't have to > > rsync folders around manually. Otherwise it is basically the same. > > > > And basically everything Ivan said =) > > > > -Zach > > > > On Tuesday, February 19, 2013 11:40:50 AM UTC-5, Ophir Michaeli wrote: > > > > > Thanks for the explanation. > > > > > > Regarding the second suggestion, is it the same as setting an "Indexing" > > > cluster, and a "Search" cluster? > > > > > > The index created by the Indexing cluster periodically copied to the > > > Search cluster folder (index) to replace the old search folder (index). > > > > > > Both this and your suggested solution has the disadvantage of not > > > allowing real time indexed data to be searchable, which is one of our > > > considerations to move to elasticsearch. > > > > > > On Tuesday, February 19, 2013 2:37:26 PM UTC+2, Ophir Michaeli wrote: > > > > > > > Hi, > > > > > > > > My question is about the best practice to divide elasticsearch indexing > > > > and search systems. > > > > > > > > Currently, without elasticsearch (and with lucene), our indexing works > > > > on several machines at one location and the search machines are at another > > > > location. > > > > > > > > The indexed data is copied or updated periodically from the indexing > > > > machines to the search machines. > > > > > > > > We want to maintain a similar structure using elasticsearch. > > > > > > > > Is it possible for the elasticsearch nodes on the indexing and search > > > > machines to be on the same cluster, > > > > > > > > so the indexing nodes will put replicas on the search nodes and will > > > > keep the replicas updated, while the search client will approach only the > > > > search nodes (ignoring the indexing nodes because they are in a different > > > > location and approaching them will slow the search). > > > > > > > > Also – is it possible to set the indexing nodes on the cluster to > > > > update the search nodes periodically (and not constantly so the search > > > > performance won't decrease)? > > > > > > > > Thanks, > > > > > > > > Ophir -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:49am UTC](https://discuss.elastic.co/t/separating-index-and-search/10798/8 "2017-07-06T02:49:49Z")