# Way to limit # of documents or storage size per node **URL:** \ **Category:** Elasticsearch\ **Created:** [August 15, 2011, 7:17pm UTC](https://discuss.elastic.co/t/way-to-limit-of-documents-or-storage-size-per-node/5165 "2011-08-15T19:17:21Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![monit06](https://avatars.discourse-cdn.com/v4/letter/m/ecd19e/32.png) [@monit06](https://discuss.elastic.co/u/monit06)\ **Post date:** [August 15, 2011, 7:17pm UTC](https://discuss.elastic.co/t/way-to-limit-of-documents-or-storage-size-per-node/5165/1 "2011-08-15T19:17:21Z")
Hi Guys, I have 2 nodes with different size of available disk, one with 500g and another with 200g.. but on both nodes I can just use 50% for elasticsearch.. Is there a way to limit # of documents or storage size per node? Cheers ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [August 15, 2011, 8:06pm UTC](https://discuss.elastic.co/t/way-to-limit-of-documents-or-storage-size-per-node/5165/2 "2011-08-15T20:06:23Z")
Hi, let me see if I can understand your question correctly: you are saying that in fact you have two servers and you can use up to 250g from the first and up to 100g from the second, correct? ES does not take account on CPU/RAM/Disk differences among cluster nodes currently. It is highly recommended to use equal nodes for optimal performance. However, if the disk space is your only concern then you can run two ES nodes on the first server, this will give you three node cluster and given that shard indices will grow more or less the same speed you can end up having three shards each up to 100g. However, this is not optimal because if you run more nodes on one physical server then if that server fails you can lose also shard replicas that were allocated to the second ES node which was running on the same physical server. Regards, Lukas On Mon, Aug 15, 2011 at 9:17 PM, monit06 [emerson.santos@gmail.com](mailto:emerson.santos@gmail.com) wrote: > Hi Guys, > > I have 2 nodes with different size of available disk, one with 500g > and another with 200g.. but on both nodes I can just use 50% for > elasticsearch.. > > Is there a way to limit # of documents or storage size per node? > > Cheers ---
**Author:** ![monit06](https://avatars.discourse-cdn.com/v4/letter/m/ecd19e/32.png) [@monit06](https://discuss.elastic.co/u/monit06)\ **Post date:** [August 15, 2011, 8:18pm UTC](https://discuss.elastic.co/t/way-to-limit-of-documents-or-storage-size-per-node/5165/3 "2011-08-15T20:18:00Z")
Hi Lukas, You are correct. My use case for ES is to load lots of log data into it as searchable documents.. currently we generate almost 5g of log per day.. so I'm trying to use all available disk space on our non-production servers. I think that performance is not currently our primary concern as searching for log data is not really done every time. So, do you recommend a different approach for that? Cheers, Emerson On Mon, Aug 15, 2011 at 10:06 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com) wrote: > Hi, > > let me see if I can understand your question correctly: you are saying that > in fact you have two servers and you can use up to 250g from the first and > up to 100g from the second, correct? > > ES does not take account on CPU/RAM/Disk differences among cluster nodes > currently. It is highly recommended to use equal nodes for optimal > performance. However, if the disk space is your only concern then you can > run two ES nodes on the first server, this will give you three node cluster > and given that shard indices will grow more or less the same speed you can > end up having three shards each up to 100g. However, this is not optimal > because if you run more nodes on one physical server then if that server > fails you can lose also shard replicas that were allocated to the second ES > node which was running on the same physical server. > > Regards, > Lukas > > On Mon, Aug 15, 2011 at 9:17 PM, monit06 [emerson.santos@gmail.com](mailto:emerson.santos@gmail.com) wrote: > > > Hi Guys, > > > > I have 2 nodes with different size of available disk, one with 500g > > and another with 200g.. but on both nodes I can just use 50% for > > elasticsearch.. > > > > Is there a way to limit # of documents or storage size per node? > > > > Cheers ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [August 15, 2011, 9:52pm UTC](https://discuss.elastic.co/t/way-to-limit-of-documents-or-storage-size-per-node/5165/4 "2011-08-15T21:52:31Z")
Hi, if you really want (has) to use 250g from one server and 100g from the other then this approach is probably to only way how to go about it for now (ie start more ES nodes on one machine). But as you can see this is not recommended for production (actually it can work pretty well until you lose one of those servers and of course more nodes on one machine will require more RAM as well). You could theoretically try to experiment with routing feature when indexing your data but I haven't tried it yet. Basically it should allow you to control which documents go to the same shard (so given that you want to split indices size in 1 vs 2 ratio you can give one routing value to every first document and second routing value to every second and third document). But that is just an dirty idea (not sure if you can control that the "bigger" shard should end up on the "bigger" machine in this case). Routing value was not meant to be used as an index size control. Regards, Lukas On Mon, Aug 15, 2011 at 10:18 PM, Emerson Santos [emerson.santos@gmail.com](mailto:emerson.santos@gmail.com)wrote: > Hi Lukas, > > You are correct. My use case for ES is to load lots of log data into it as > searchable documents.. currently we generate almost 5g of log per day.. so > I'm trying to use all available disk space on our non-production servers. > > I think that performance is not currently our primary concern as searching > for log data is not really done every time. So, do you recommend a different > approach for that? > > Cheers, > Emerson > > On Mon, Aug 15, 2011 at 10:06 PM, Lukáš Vlček [lukas.vlcek@gmail.com](mailto:lukas.vlcek@gmail.com)wrote: > > > Hi, > > > > let me see if I can understand your question correctly: you are saying > > that in fact you have two servers and you can use up to 250g from the first > > and up to 100g from the second, correct? > > > > ES does not take account on CPU/RAM/Disk differences among cluster nodes > > currently. It is highly recommended to use equal nodes for optimal > > performance. However, if the disk space is your only concern then you can > > run two ES nodes on the first server, this will give you three node cluster > > and given that shard indices will grow more or less the same speed you can > > end up having three shards each up to 100g. However, this is not optimal > > because if you run more nodes on one physical server then if that server > > fails you can lose also shard replicas that were allocated to the second ES > > node which was running on the same physical server. > > > > Regards, > > Lukas > > > > On Mon, Aug 15, 2011 at 9:17 PM, monit06 [emerson.santos@gmail.com](mailto:emerson.santos@gmail.com)wrote: > > > > > Hi Guys, > > > > > > I have 2 nodes with different size of available disk, one with 500g > > > and another with 200g.. but on both nodes I can just use 50% for > > > elasticsearch.. > > > > > > Is there a way to limit # of documents or storage size per node? > > > > > > Cheers ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:56am UTC](https://discuss.elastic.co/t/way-to-limit-of-documents-or-storage-size-per-node/5165/5 "2017-07-06T03:56:53Z")