# Why not have maximum replication **URL:** \ **Category:** Elasticsearch\ **Created:** [November 17, 2014, 8:11pm UTC](https://discuss.elastic.co/t/why-not-have-maximum-replication/20797 "2014-11-17T20:11:15Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Christopher\_Ambler\_2](https://avatars.discourse-cdn.com/v4/letter/c/b5ac83/32.png) [@Christopher\_Ambler\_2](https://discuss.elastic.co/u/Christopher_Ambler_2)\ **Post date:** [November 17, 2014, 8:11pm UTC](https://discuss.elastic.co/t/why-not-have-maximum-replication/20797/1 "2014-11-17T20:11:15Z")
Can someone explain to me why, if disk space is not an issue, I don't want maximum replication such that every node has every shard? It seems to me that there would be no real downside here as long as I'm not worried about filling up a disk and my updates happen infrequently and in a timely manner. Am I missing the obvious? πŸ˜‰ -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/897baef9-225e-450f-ab5d-a114ff54be9d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/897baef9-225e-450f-ab5d-a114ff54be9d%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Jure\_Koren](https://avatars.discourse-cdn.com/v4/letter/j/8797f3/32.png) [@Jure\_Koren](https://discuss.elastic.co/u/Jure_Koren)\ **Post date:** [November 17, 2014, 8:21pm UTC](https://discuss.elastic.co/t/why-not-have-maximum-replication/20797/2 "2014-11-17T20:21:11Z")
On Monday 17 of November 2014 12:11:15 Christopher Ambler wrote: > Can someone explain to me why, if disk space is not an issue, I don't want > maximum replication such that every node has every shard? > > It seems to me that there would be no real downside here as long as I'm not > worried about filling up a disk and my updates happen infrequently and in a > timely manner. > > Am I missing the obvious? πŸ˜‰ Your filesystem cache can only hold so much. If you're hitting disks often, performance will suffer a lot. Best, -- Jure Koren -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1713599.GklGf3b9bn%40havelock](https://groups.google.com/d/msgid/elasticsearch/1713599.GklGf3b9bn%40havelock). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Christopher\_Ambler\_2](https://avatars.discourse-cdn.com/v4/letter/c/b5ac83/32.png) [@Christopher\_Ambler\_2](https://discuss.elastic.co/u/Christopher_Ambler_2)\ **Post date:** [November 18, 2014, 7:12am UTC](https://discuss.elastic.co/t/why-not-have-maximum-replication/20797/3 "2014-11-18T07:12:26Z")
I don't see how that's an issue. Data needs to be stored somewhere. If I'm hitting disk often, I'm going to do so no matter what, no matter where. So why not have the data on all nodes? On Monday, November 17, 2014 12:11:15 PM UTC-8, Christopher Ambler wrote: > Can someone explain to me why, if disk space is not an issue, I don't want > maximum replication such that every node has every shard? > > It seems to me that there would be no real downside here as long as I'm > not worried about filling up a disk and my updates happen infrequently and > in a timely manner. > > Am I missing the obvious? πŸ˜‰ -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6e54e3dc-c9f3-4428-977e-fc3f0e64da8d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6e54e3dc-c9f3-4428-977e-fc3f0e64da8d%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [November 18, 2014, 7:17am UTC](https://discuss.elastic.co/t/why-not-have-maximum-replication/20797/4 "2014-11-18T07:17:23Z")
There's a difference in FS cache and then do an actual read from the FS, the latter being a lot slower. If you have SSDs, then this might be feasible. But overall you're potentially wasting a lot of resources. On 18 November 2014 18:12, Christopher Ambler [const.dogberry@gmail.com](mailto:const.dogberry@gmail.com) wrote: > I don't see how that's an issue. > > Data needs to be stored somewhere. If I'm hitting disk often, I'm going to > do so no matter what, no matter where. > > So why not have the data on all nodes? > > On Monday, November 17, 2014 12:11:15 PM UTC-8, Christopher Ambler wrote: > > > Can someone explain to me why, if disk space is not an issue, I don't > > want maximum replication such that every node has every shard? > > > > It seems to me that there would be no real downside here as long as I'm > > not worried about filling up a disk and my updates happen infrequently and > > in a timely manner. > > > > Am I missing the obvious? πŸ˜‰ > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/6e54e3dc-c9f3-4428-977e-fc3f0e64da8d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6e54e3dc-c9f3-4428-977e-fc3f0e64da8d%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/6e54e3dc-c9f3-4428-977e-fc3f0e64da8d%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6e54e3dc-c9f3-4428-977e-fc3f0e64da8d%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZmE-nkPFoU%2BgsKFw9r7aL33%3De%2B9qfHMSOJGvmxxsTn2mA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZmE-nkPFoU%2BgsKFw9r7aL33%3De%2B9qfHMSOJGvmxxsTn2mA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Jure\_Koren](https://avatars.discourse-cdn.com/v4/letter/j/8797f3/32.png) [@Jure\_Koren](https://discuss.elastic.co/u/Jure_Koren)\ **Post date:** [November 18, 2014, 9:14am UTC](https://discuss.elastic.co/t/why-not-have-maximum-replication/20797/5 "2014-11-18T09:14:33Z")
On Monday 17 of November 2014 23:12:26 Christopher Ambler wrote: > I don't see how that's an issue. > > Data needs to be stored somewhere. If I'm hitting disk often, I'm going to > do so no matter what, no matter where. > > So why not have the data on all nodes? Because your nodes usually only have enough memory to store a limited subset of your index. If nodes store all shards, they have to keep parts of all shards in memory for these to be queried efficiently. If you have enough memory, that will usually work fine. If there is a way to keep shards stored, but prevent elasticsearch from querying them (or otherwise trying to read them from disk), I'd sure like to know about it, because it would make a great way to do backups or recovery from a "passive" node without disrupting the file system cache on active nodes under load. -- Jure Koren -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1799499.UjryPpiTIh%40coin](https://groups.google.com/d/msgid/elasticsearch/1799499.UjryPpiTIh%40coin). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Duncan\_Innes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/duncan_innes/32/533_2.png) [@Duncan\_Innes](https://discuss.elastic.co/u/Duncan_Innes)\ **Post date:** [November 18, 2014, 9:36am UTC](https://discuss.elastic.co/t/why-not-have-maximum-replication/20797/6 "2014-11-18T09:36:46Z")
Depends on how many nodes you have of course, but if you go for a replica on every node, your write performance will take a hit. So high throughput logging will be difficult. I see the point though - _if_ you have the performance for it and _if_ you're not trying to log at huge rates, then there is definitely extra security in a relpica on every node. Is it worth it? Up to you really. We currently have 5 nodes with 2 replicas on every index. This allows us to lose any 2 machines in the cluster and keep our heads above water. This is a reasonable state of affairs for us. Our set up is a bit like a Raid 5 disk array, whereas you are looking at Raid 1. Horses for courses I guess. D On Monday, 17 November 2014 20:11:15 UTC, Christopher Ambler wrote: > Can someone explain to me why, if disk space is not an issue, I don't want > maximum replication such that every node has every shard? > > It seems to me that there would be no real downside here as long as I'm > not worried about filling up a disk and my updates happen infrequently and > in a timely manner. > > Am I missing the obvious? πŸ˜‰ -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/045403f7-a4fe-44bf-b1c5-88a27d1c01f5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/045403f7-a4fe-44bf-b1c5-88a27d1c01f5%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [November 18, 2014, 9:54am UTC](https://discuss.elastic.co/t/why-not-have-maximum-replication/20797/7 "2014-11-18T09:54:54Z")
Maximum replication kills write performance. JΓΆrg On Mon, Nov 17, 2014 at 9:11 PM, Christopher Ambler \< [const.dogberry@gmail.com](mailto:const.dogberry@gmail.com)\> wrote: > Can someone explain to me why, if disk space is not an issue, I don't want > maximum replication such that every node has every shard? > > It seems to me that there would be no real downside here as long as I'm > not worried about filling up a disk and my updates happen infrequently and > in a timely manner. > > Am I missing the obvious? πŸ˜‰ > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/897baef9-225e-450f-ab5d-a114ff54be9d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/897baef9-225e-450f-ab5d-a114ff54be9d%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/897baef9-225e-450f-ab5d-a114ff54be9d%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/897baef9-225e-450f-ab5d-a114ff54be9d%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFfnum81%2Bzfej\_G4m3wQ\_ZoeO5JR\_kYFqjfqdAK-E9J2g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFfnum81%2Bzfej_G4m3wQ_ZoeO5JR_kYFqjfqdAK-E9J2g%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:49am UTC](https://discuss.elastic.co/t/why-not-have-maximum-replication/20797/8 "2017-07-06T00:49:18Z")