# Read\\Write paths **URL:** \ **Category:** Elasticsearch\ **Created:** [October 13, 2010, 1:58pm UTC](https://discuss.elastic.co/t/read-write-paths/3437 "2010-10-13T13:58:33Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Keidi](https://avatars.discourse-cdn.com/v4/letter/k/3d9bf3/32.png) [@Keidi](https://discuss.elastic.co/u/Keidi)\ **Post date:** [October 13, 2010, 1:58pm UTC](https://discuss.elastic.co/t/read-write-paths/3437/1 "2010-10-13T13:58:33Z")
Hi, I'm new to ElasticSearch but from what I have seen so far I couldn't figure out whether it can support a read\write path architecture. Thanks! ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [October 13, 2010, 2:39pm UTC](https://discuss.elastic.co/t/read-write-paths/3437/2 "2010-10-13T14:39:18Z")
Hi, if you mean search or get for the read path then you might want to check GET REST API [http://www.elasticsearch.com/docs/elasticsearch/rest\_api/get/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/get/) or search REST API [http://www.elasticsearch.com/docs/elasticsearch/rest\_api/search/uri\_request/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/uri_request/) (note there is also more advanced search REST API here [http://www.elasticsearch.com/docs/elasticsearch/rest\_api/search/body\_request/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/body_request/) ) if you mean indexing documents for the write path then you can check index REST API [http://www.elasticsearch.com/docs/elasticsearch/rest\_api/index/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/index/) (or bulk updates [http://www.elasticsearch.com/docs/elasticsearch/rest\_api/bulk/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/bulk/) ). But there is much more to this... what exactly are you after? Regards, Lukas On Wed, Oct 13, 2010 at 3:58 PM, Keidi [shahar.clf@gmail.com](mailto:shahar.clf@gmail.com) wrote: > Hi, > > I'm new to Elasticsearch but from what I have seen so far I couldn't > figure out whether it can support a read\write path architecture. > > Thanks! ---
**Author:** ![Keidi](https://avatars.discourse-cdn.com/v4/letter/k/3d9bf3/32.png) [@Keidi](https://discuss.elastic.co/u/Keidi)\ **Post date:** [October 14, 2010, 10:24am UTC](https://discuss.elastic.co/t/read-write-paths/3437/3 "2010-10-14T10:24:00Z")
Hi Lukas, I believe you didn't follow my meaning. When I say read/write paths I mean separating between nodes that act as write-only nodes (write path) and nodes that are read-only nodes (read path). The idea is to reduce as much as possible the load off the read path for better query performance. In other words - we don't want heavy indexing requests to take resources away from query requests. This of course requires some mechanism that efficiently and regularly replicates data from the write-path to the read-path. I'm guessing this is not implemented by ElasticSearch but it can be implemented by us (for example, by using different indexes for the different paths and migrating data in bulks). The question is: does this kind of approach goes against everything ElasticSearch stands for? I'm not sure, but I did get the feeling that the whole idea is to solve performance problems not by separating between read\write paths but by scaling-out (i.e. adding more machines). ## Thanks, Shahar View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Read-Write-paths-tp1694631p1700006.html](http://elasticsearch-users.115913.n3.nabble.com/Read-Write-paths-tp1694631p1700006.html) Sent from the ElasticSearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![Keidi](https://avatars.discourse-cdn.com/v4/letter/k/3d9bf3/32.png) [@Keidi](https://discuss.elastic.co/u/Keidi)\ **Post date:** [October 14, 2010, 12:31pm UTC](https://discuss.elastic.co/t/read-write-paths/3437/4 "2010-10-14T12:31:49Z")
Hi Lukas, I believe you didn't follow my meaning. When I say read/write paths I mean separating between nodes that act as write-only nodes (write path) and nodes that are read-only nodes (read path). The idea is to reduce as much as possible the load off the read path for better query performance. In other words - we don't want heavy indexing requests to take resources away from query requests. This of course requires some mechanism that efficiently and regularly replicates data from the write-path to the read-path. I'm guessing this is not implemented by ElasticSearch but it can be implemented by us (for example, by using different indexes for the different paths and migrating data in bulks). The question is: does this kind of approach goes against everything ElasticSearch stands for? I'm not sure, but I did get the feeling that the whole idea is to solve performance problems not by separating between read\write paths but by scaling-out (i.e. adding more machines). ## Thanks, Shahar View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Read-Write-paths-tp1694631p1700695.html](http://elasticsearch-users.115913.n3.nabble.com/Read-Write-paths-tp1694631p1700695.html) Sent from the ElasticSearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![Keidi](https://avatars.discourse-cdn.com/v4/letter/k/3d9bf3/32.png) [@Keidi](https://discuss.elastic.co/u/Keidi)\ **Post date:** [October 14, 2010, 1:01pm UTC](https://discuss.elastic.co/t/read-write-paths/3437/5 "2010-10-14T13:01:45Z")
Hi Lukas, I believe you didn't follow my meaning. When I say read/write paths I mean separating between nodes that act as write-only nodes (write path) and nodes that are read-only nodes (read path). The idea is to reduce as much as possible the load off the read path for better query performance. In other words - we don't want heavy indexing requests to take resources away from query requests. This of course requires some mechanism that efficiently and regularly replicates data from the write-path to the read-path. I'm guessing this is not implemented by Elasticsearch but it can be implemented by us (for example, by using different indexes for the different paths and migrating data in bulks). The question is: does this kind of approach goes against everything Elasticsearch stands for? I'm not sure, but I did get the feeling that the whole idea is to solve performance problems not by separating between read\write paths but by scaling-out (i.e. adding more machines). Thanks, Shahar On Oct 13, 4:39 pm, Lukáš Vlček [lukas.vl...@gmail.com](mailto:lukas.vl...@gmail.com) wrote: > Hi, > > if you mean search or get for the read path then you might want to check GET > REST APIhttp://www.elasticsearch.com/docs/elasticsearch/rest\_api/get/or > search REST APIhttp://www.elasticsearch.com/docs/elasticsearch/rest\_api/search/uri\_r... > (note > there is also more advanced search REST API herehttp://www.elasticsearch.com/docs/elasticsearch/rest\_api/search/body\_... > ) > > if you mean indexing documents for the write path then you can check index > REST APIhttp://www.elasticsearch.com/docs/elasticsearch/rest\_api/index/(or > bulk updateshttp://www.elasticsearch.com/docs/elasticsearch/rest\_api/bulk/ > ). > > But there is much more to this... what exactly are you after? > > Regards, > Lukas > > On Wed, Oct 13, 2010 at 3:58 PM, Keidi [shahar....@gmail.com](mailto:shahar....@gmail.com) wrote: > > > Hi, > > > I'm new to Elasticsearch but from what I have seen so far I couldn't > > figure out whether it can support a read\write path architecture. > > > Thanks! ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 14, 2010, 1:21pm UTC](https://discuss.elastic.co/t/read-write-paths/3437/6 "2010-10-14T13:21:21Z")
Hi, Yea, this is not how elasticsearch works. Basically, index operation is replicated to all the shard replicas, and executed there as well. More replicas will give you better search scalability, but they will also do indexing. I think what you mean is something like periodically pulling the changed lucene index to "read only" replicas so they will be searched on, certainly something that can be done, but does go against the current model of (near) real time, atomicity, and others. But this model does incur its overhead and has downsides, both in terms of its freshness (how often do you commit lucene in order to get those changes in), and its periodic load on nodes for moving data around (which can be substantial). Is there a chance that there is a premature optimization here, have you really seen problems? -shay.banon On Thu, Oct 14, 2010 at 3:01 PM, Keidi [shahar.clf@gmail.com](mailto:shahar.clf@gmail.com) wrote: > Hi Lukas, > > I believe you didn't follow my meaning. When I say read/write paths I > mean separating between nodes that act as write-only nodes (write > path) and nodes that are read-only nodes (read path). The idea is to > reduce as much as possible the load off the read path for better query > performance. In other words - we don't want heavy indexing requests to > take resources away from query requests. > This of course requires some mechanism that efficiently and regularly > replicates data from the write-path to the read-path. I'm guessing > this is not implemented by Elasticsearch but it can be implemented by > us (for example, by using different indexes for the different paths > and migrating data in bulks). > The question is: does this kind of approach goes against everything > Elasticsearch stands for? I'm not sure, but I did get the feeling that > the whole idea is to solve performance problems not by separating > between read\write paths but by scaling-out (i.e. adding more > machines). > > Thanks, > Shahar > > On Oct 13, 4:39 pm, Lukáš Vlček [lukas.vl...@gmail.com](mailto:lukas.vl...@gmail.com) wrote: > > > Hi, > > > > if you mean search or get for the read path then you might want to check > > GET > > REST APIhttp://www.elasticsearch.com/docs/elasticsearch/rest\_api/get/or > > search REST APIhttp:// > > [www.elasticsearch.com/docs/elasticsearch/rest\_api/search/uri\_r](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/uri_r)... > > (note > > there is also more advanced search REST API herehttp:// > > [www.elasticsearch.com/docs/elasticsearch/rest\_api/search/body\_](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/body_)... > > ) > > > > if you mean indexing documents for the write path then you can check > > index > > REST APIhttp:// > > [www.elasticsearch.com/docs/elasticsearch/rest\_api/index/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/index/)(or > > bulk updateshttp:// > > [www.elasticsearch.com/docs/elasticsearch/rest\_api/bulk/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/bulk/) > > ). > > > > But there is much more to this... what exactly are you after? > > > > Regards, > > Lukas > > > > On Wed, Oct 13, 2010 at 3:58 PM, Keidi [shahar....@gmail.com](mailto:shahar....@gmail.com) wrote: > > > > > Hi, > > > > > I'm new to Elasticsearch but from what I have seen so far I couldn't > > > figure out whether it can support a read\write path architecture. > > > > > Thanks! ---
**Author:** ![Keidi](https://avatars.discourse-cdn.com/v4/letter/k/3d9bf3/32.png) [@Keidi](https://discuss.elastic.co/u/Keidi)\ **Post date:** [October 14, 2010, 2:45pm UTC](https://discuss.elastic.co/t/read-write-paths/3437/7 "2010-10-14T14:45:33Z")
Hi Shay, We haven't seen real performance problems. To tell the truth, we still have not been able to overload Elasticsearch (and we were using big guns :)). But we are still only experimenting. The reason I raised this issue is that using read\write paths was one of the architectural options for our project. I just wanted to make sure that with Elasticsearch this kind of architecture is against design. I'm positive that in case we choose ES we will not be using read\write paths. Thanks for the answer, Shahar On Oct 14, 3:21 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > Hi, > > Yea, this is not how elasticsearch works. Basically, index operation is > replicated to all the shard replicas, and executed there as well. More > replicas will give you better search scalability, but they will also do > indexing. I think what you mean is something like periodically pulling the > changed lucene index to "read only" replicas so they will be searched on, > certainly something that can be done, but does go against the current model > of (near) real time, atomicity, and others. > > But this model does incur its overhead and has downsides, both in terms > of its freshness (how often do you commit lucene in order to get those > changes in), and its periodic load on nodes for moving data around (which > can be substantial). > > Is there a chance that there is a premature optimization here, have you > really seen problems? > > -shay.banon > > On Thu, Oct 14, 2010 at 3:01 PM, Keidi [shahar....@gmail.com](mailto:shahar....@gmail.com) wrote: > > > Hi Lukas, > > > I believe you didn't follow my meaning. When I say read/write paths I > > mean separating between nodes that act as write-only nodes (write > > path) and nodes that are read-only nodes (read path). The idea is to > > reduce as much as possible the load off the read path for better query > > performance. In other words - we don't want heavy indexing requests to > > take resources away from query requests. > > This of course requires some mechanism that efficiently and regularly > > replicates data from the write-path to the read-path. I'm guessing > > this is not implemented by Elasticsearch but it can be implemented by > > us (for example, by using different indexes for the different paths > > and migrating data in bulks). > > The question is: does this kind of approach goes against everything > > Elasticsearch stands for? I'm not sure, but I did get the feeling that > > the whole idea is to solve performance problems not by separating > > between read\write paths but by scaling-out (i.e. adding more > > machines). > > > Thanks, > > Shahar > > > On Oct 13, 4:39 pm, Lukáš Vlček [lukas.vl...@gmail.com](mailto:lukas.vl...@gmail.com) wrote: > > > > > Hi, > > > > if you mean search or get for the read path then you might want to check > > > GET > > > REST APIhttp://www.elasticsearch.com/docs/elasticsearch/rest\_api/get/or > > > search REST APIhttp:// > > > [www.elasticsearch.com/docs/elasticsearch/rest\_api/search/uri\_r](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/uri_r)... > > > (note > > > there is also more advanced search REST API herehttp:// > > > [www.elasticsearch.com/docs/elasticsearch/rest\_api/search/body\_](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/body_)... > > > ) > > > > if you mean indexing documents for the write path then you can check > > > index > > > REST APIhttp:// > > > [www.elasticsearch.com/docs/elasticsearch/rest\_api/index/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/index/)(or > > > bulk updateshttp:// > > > [www.elasticsearch.com/docs/elasticsearch/rest\_api/bulk/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/bulk/) > > > ). > > > > But there is much more to this... what exactly are you after? > > > > Regards, > > > Lukas > > > > On Wed, Oct 13, 2010 at 3:58 PM, Keidi [shahar....@gmail.com](mailto:shahar....@gmail.com) wrote: > > > > > > > Hi, > > > > > I'm new to Elasticsearch but from what I have seen so far I couldn't > > > > figure out whether it can support a read\write path architecture. > > > > > Thanks! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:18am UTC](https://discuss.elastic.co/t/read-write-paths/3437/8 "2017-07-06T04:18:01Z")