# 20 shards per 5 nodes, thoughts **URL:** \ **Category:** Elasticsearch\ **Created:** [November 17, 2014, 9:02am UTC](https://discuss.elastic.co/t/20-shards-per-5-nodes-thoughts/20777 "2014-11-17T09:02:43Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Pavel\_P](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pavel_p/32/1092_2.png) [@Pavel\_P](https://discuss.elastic.co/u/Pavel_P)\ **Post date:** [November 17, 2014, 9:02am UTC](https://discuss.elastic.co/t/20-shards-per-5-nodes-thoughts/20777/1 "2014-11-17T09:02:43Z")
Hi, I have a cluster from 5 nodes, where I store the information from the logstash. Recently I've tried to increase the number of shards in the logstash index to 20 (from 5). From the beginning everything went well, all the shards were allocated and the cluster state was green. But, currently, when the new index is started (at the beginning of the day), I met the situation when some shards are not allocated: [https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png](https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png) Nothing happens during the day, while I thing the cluster has resources. If I would restart some nodes from the cluster, it could turn out that all the shards would be allocated. The idea which I'm trying to reach - more shards =\> each shard is smaller =\> it would allocate them faster, it would index faster, because of the indexing on each shard. Questions: 1. Why it is not able to allocate those shards during the index creation? 2. Why it does not allocate those shards during the day? 3. What is the recommended shards number for N nodes? Any thoughts are welcome. Regards, -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/cefe191d-c71a-4fa9-8432-092315b1b7df%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cefe191d-c71a-4fa9-8432-092315b1b7df%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Pavel\_P](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pavel_p/32/1092_2.png) [@Pavel\_P](https://discuss.elastic.co/u/Pavel_P)\ **Post date:** [November 18, 2014, 8:59am UTC](https://discuss.elastic.co/t/20-shards-per-5-nodes-thoughts/20777/2 "2014-11-18T08:59:25Z")
Hi again, The question is still actual. Yesterday I've updated the ES version to 1.4.0, in the night new index was created, currently it is not able to assign 2 nodes: [https://lh5.googleusercontent.com/-e5DZN7BvA3w/VGsKW-0Su2I/AAAAAAAAAKo/lrst5q10vWI/s1600/Screen%2BShot%2B2014-11-18%2Bat%2B10.58.37%2BAM.png](https://lh5.googleusercontent.com/-e5DZN7BvA3w/VGsKW-0Su2I/AAAAAAAAAKo/lrst5q10vWI/s1600/Screen%2BShot%2B2014-11-18%2Bat%2B10.58.37%2BAM.png) Please, advice me something 🙂 On Monday, November 17, 2014 11:02:43 AM UTC+2, Pavel P wrote: > Hi, > > I have a cluster from 5 nodes, where I store the information from the > logstash. > Recently I've tried to increase the number of shards in the logstash index > to 20 (from 5). > > From the beginning everything went well, all the shards were allocated and > the cluster state was green. > > But, currently, when the new index is started (at the beginning of the > day), I met the situation when some shards are not allocated: > > [https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png](https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png) > > Nothing happens during the day, while I thing the cluster has resources. > > If I would restart some nodes from the cluster, it could turn out that all > the shards would be allocated. > > The idea which I'm trying to reach - more shards =\> each shard is smaller > =\> it would allocate them faster, it would index faster, because of the > indexing on each shard. > > Questions: > > 1. Why it is not able to allocate those shards during the index creation? > 2. Why it does not allocate those shards during the day? > 3. What is the recommended shards number for N nodes? > > Any thoughts are welcome. > > Regards, -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6b9b0a09-c6b3-428d-ae26-148e94480a16%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6b9b0a09-c6b3-428d-ae26-148e94480a16%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [November 18, 2014, 9:26am UTC](https://discuss.elastic.co/t/20-shards-per-5-nodes-thoughts/20777/3 "2014-11-18T09:26:19Z")
Your logic around having so many shards may make sense, but you still end up having to move the same amount of data and it's more over head. You should have N number of shards, where N is the number of nodes. Check your logs, there may be something in there. Otherwise can you provide more details on your node size; RAM, disk, CPU, ES and java version, that sort of thing, On 18 November 2014 19:59, Pavel P [pavel@kredito.de](mailto:pavel@kredito.de) wrote: > Hi again, > > The question is still actual. > > Yesterday I've updated the ES version to 1.4.0, in the night new index was > created, currently it is not able to assign 2 nodes: > > [https://lh5.googleusercontent.com/-e5DZN7BvA3w/VGsKW-0Su2I/AAAAAAAAAKo/lrst5q10vWI/s1600/Screen%2BShot%2B2014-11-18%2Bat%2B10.58.37%2BAM.png](https://lh5.googleusercontent.com/-e5DZN7BvA3w/VGsKW-0Su2I/AAAAAAAAAKo/lrst5q10vWI/s1600/Screen%2BShot%2B2014-11-18%2Bat%2B10.58.37%2BAM.png) > > Please, advice me something 🙂 > > On Monday, November 17, 2014 11:02:43 AM UTC+2, Pavel P wrote: > > > Hi, > > > > I have a cluster from 5 nodes, where I store the information from the > > logstash. > > Recently I've tried to increase the number of shards in the logstash > > index to 20 (from 5). > > > > From the beginning everything went well, all the shards were allocated > > and the cluster state was green. > > > > But, currently, when the new index is started (at the beginning of the > > day), I met the situation when some shards are not allocated: > > > > [https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png](https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png) > > > > Nothing happens during the day, while I thing the cluster has resources. > > > > If I would restart some nodes from the cluster, it could turn out that > > all the shards would be allocated. > > > > The idea which I'm trying to reach - more shards =\> each shard is smaller > > =\> it would allocate them faster, it would index faster, because of the > > indexing on each shard. > > > > Questions: > > > > 1. Why it is not able to allocate those shards during the index creation? > > 2. Why it does not allocate those shards during the day? > > 3. What is the recommended shards number for N nodes? > > > > Any thoughts are welcome. > > > > Regards, > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/6b9b0a09-c6b3-428d-ae26-148e94480a16%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6b9b0a09-c6b3-428d-ae26-148e94480a16%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/6b9b0a09-c6b3-428d-ae26-148e94480a16%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6b9b0a09-c6b3-428d-ae26-148e94480a16%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZkmUcML7MGPvf\_93xVhVsQR4X5iETUpquDbmkFpo3nMXw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZkmUcML7MGPvf_93xVhVsQR4X5iETUpquDbmkFpo3nMXw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Pavel\_P](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pavel_p/32/1092_2.png) [@Pavel\_P](https://discuss.elastic.co/u/Pavel_P)\ **Post date:** [November 18, 2014, 9:39am UTC](https://discuss.elastic.co/t/20-shards-per-5-nodes-thoughts/20777/4 "2014-11-18T09:39:13Z")
Unfortunately there is nothing in the logs about the issue. Here is the hardware: [image: Inline image 1] The ES version is 1.4.0 , when I reduce the number of replicas to 0, it would became green. Here > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. the next is stated: > If a shard cannot be assigned, for example you’ve overallocated the number > of replicas for the number of nodes in the cluster, they will remain > UNASSIGNED. How can I understand the real reason of unassigning the shards? According to the bigdesk - the nodes are feeling themselves very good, there are no queues and enough of the resources. [image: Inline image 2] I think that my hardware can hold more then 5 nodes. I've tried once to reduce them to 10, but during the index creation some were still unassigned. However, when I do restart of some node - it could be that every would be assigned after the shards relocation process. Regards, On Tue, Nov 18, 2014 at 11:26 AM, Mark Walkom [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > Your logic around having so many shards may make sense, but you still end > up having to move the same amount of data and it's more over head. > You should have N number of shards, where N is the number of nodes. > > Check your logs, there may be something in there. Otherwise can you > provide more details on your node size; RAM, disk, CPU, ES and java > version, that sort of thing, > > On 18 November 2014 19:59, Pavel P [pavel@kredito.de](mailto:pavel@kredito.de) wrote: > > > Hi again, > > > > The question is still actual. > > > > Yesterday I've updated the ES version to 1.4.0, in the night new index > > was created, currently it is not able to assign 2 nodes: > > > > [https://lh5.googleusercontent.com/-e5DZN7BvA3w/VGsKW-0Su2I/AAAAAAAAAKo/lrst5q10vWI/s1600/Screen%2BShot%2B2014-11-18%2Bat%2B10.58.37%2BAM.png](https://lh5.googleusercontent.com/-e5DZN7BvA3w/VGsKW-0Su2I/AAAAAAAAAKo/lrst5q10vWI/s1600/Screen%2BShot%2B2014-11-18%2Bat%2B10.58.37%2BAM.png) > > > > Please, advice me something 🙂 > > > > On Monday, November 17, 2014 11:02:43 AM UTC+2, Pavel P wrote: > > > > > Hi, > > > > > > I have a cluster from 5 nodes, where I store the information from the > > > logstash. > > > Recently I've tried to increase the number of shards in the logstash > > > index to 20 (from 5). > > > > > > From the beginning everything went well, all the shards were allocated > > > and the cluster state was green. > > > > > > But, currently, when the new index is started (at the beginning of the > > > day), I met the situation when some shards are not allocated: > > > > > > [https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png](https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png) > > > > > > Nothing happens during the day, while I thing the cluster has resources. > > > > > > If I would restart some nodes from the cluster, it could turn out that > > > all the shards would be allocated. > > > > > > The idea which I'm trying to reach - more shards =\> each shard is > > > smaller =\> it would allocate them faster, it would index faster, because of > > > the indexing on each shard. > > > > > > Questions: > > > > > > 1. Why it is not able to allocate those shards during the index creation? > > > 2. Why it does not allocate those shards during the day? > > > 3. What is the recommended shards number for N nodes? > > > > > > Any thoughts are welcome. > > > > > > Regards, > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/6b9b0a09-c6b3-428d-ae26-148e94480a16%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6b9b0a09-c6b3-428d-ae26-148e94480a16%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/6b9b0a09-c6b3-428d-ae26-148e94480a16%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6b9b0a09-c6b3-428d-ae26-148e94480a16%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/q5Xkn9pc9EM/unsubscribe](https://groups.google.com/d/topic/elasticsearch/q5Xkn9pc9EM/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZkmUcML7MGPvf\_93xVhVsQR4X5iETUpquDbmkFpo3nMXw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZkmUcML7MGPvf_93xVhVsQR4X5iETUpquDbmkFpo3nMXw%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZkmUcML7MGPvf\_93xVhVsQR4X5iETUpquDbmkFpo3nMXw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZkmUcML7MGPvf_93xVhVsQR4X5iETUpquDbmkFpo3nMXw%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- _Pavel Polyakov_ Software Engineer - PHP E-mail: [pavel@kredito.de](mailto:pavel@kredito.de) Skype: pavel.polyakov.x1 [https://www.facebook.com/kreditech](https://www.facebook.com/kreditech) Kreditech Holding SSL GmbH Am Sandtorkai 50, 20457 Hamburg, Germany Office phone: +49 (0)40 - 605905-60 Authorized representatives: Sebastian Diemer, Alexander Graubner-Müller Company registration: Hamburg HRB122027 [www.kreditech.com](http://www.kreditech.com) [Kreditech](http://facebook.com/kreditech) [https://www.facebook.com/kreditech](https://www.facebook.com/kreditech) [https://www.facebook.com/kreditech](https://www.facebook.com/kreditech) This e-mail contains confidential and/or legally protected information. If you are not the intended recipient or if you have received this e-mail by error please notify the sender immediately and destroy this e-mail. Any unauthorized review, copying, disclosure or distribution of the material in this e-mail is strictly forbidden. The contents of this e-mail is legally binding only if it is confirmed by letter or fax. The sending of e-mails to us does not have any period-protecting effect. Thank you for your cooperation. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAFVUaqNLQs05aONd8dOWcM4GWh1K8kKk-Ec6iMopCYwxaS%2BLtQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAFVUaqNLQs05aONd8dOWcM4GWh1K8kKk-Ec6iMopCYwxaS%2BLtQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Pavel\_P](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pavel_p/32/1092_2.png) [@Pavel\_P](https://discuss.elastic.co/u/Pavel_P)\ **Post date:** [November 18, 2014, 1:17pm UTC](https://discuss.elastic.co/t/20-shards-per-5-nodes-thoughts/20777/5 "2014-11-18T13:17:02Z")
It looks, that I have 2 shards unassigned, because two other shards were stuck in "RELOCATING" state. [https://lh5.googleusercontent.com/-TD84Uz8lg5Q/VGtGVXVIFVI/AAAAAAAAAK4/ITudefzX1qQ/s1600/ES\_RELOCATING.png](https://lh5.googleusercontent.com/-TD84Uz8lg5Q/VGtGVXVIFVI/AAAAAAAAAK4/ITudefzX1qQ/s1600/ES_RELOCATING.png) And I think I have them migrating here and there, and while they are trying to find the new home for themselves, it's not possible to allocate those 2 shards. I've tried to close the index and then open - when the index was closed - all the remaining shards were allocated successfully, after I've opened it again - no shards remain unassigned. So the current question is - why some shards are relocating constantly. Regards, On Monday, November 17, 2014 11:02:43 AM UTC+2, Pavel P wrote: > Hi, > > I have a cluster from 5 nodes, where I store the information from the > logstash. > Recently I've tried to increase the number of shards in the logstash index > to 20 (from 5). > > From the beginning everything went well, all the shards were allocated and > the cluster state was green. > > But, currently, when the new index is started (at the beginning of the > day), I met the situation when some shards are not allocated: > > [https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png](https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png) > > Nothing happens during the day, while I thing the cluster has resources. > > If I would restart some nodes from the cluster, it could turn out that all > the shards would be allocated. > > The idea which I'm trying to reach - more shards =\> each shard is smaller > =\> it would allocate them faster, it would index faster, because of the > indexing on each shard. > > Questions: > > 1. Why it is not able to allocate those shards during the index creation? > 2. Why it does not allocate those shards during the day? > 3. What is the recommended shards number for N nodes? > > Any thoughts are welcome. > > Regards, -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f5faeb78-a7ee-48b9-a0c0-c42e68b05cb1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f5faeb78-a7ee-48b9-a0c0-c42e68b05cb1%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Jakub\_Podeszwik](https://avatars.discourse-cdn.com/v4/letter/j/a3d4f5/32.png) [@Jakub\_Podeszwik](https://discuss.elastic.co/u/Jakub_Podeszwik)\ **Post date:** [November 19, 2014, 7:09am UTC](https://discuss.elastic.co/t/20-shards-per-5-nodes-thoughts/20777/6 "2014-11-19T07:09:56Z")
There should be some info in your logs about checksum errors or something like that. Relocation fi On Tuesday, 18 November 2014 14:17:02 UTC+1, Pavel P wrote: > It looks, that I have 2 shards unassigned, because two other shards were > stuck in "RELOCATING" state. > > [https://lh5.googleusercontent.com/-TD84Uz8lg5Q/VGtGVXVIFVI/AAAAAAAAAK4/ITudefzX1qQ/s1600/ES\_RELOCATING.png](https://lh5.googleusercontent.com/-TD84Uz8lg5Q/VGtGVXVIFVI/AAAAAAAAAK4/ITudefzX1qQ/s1600/ES_RELOCATING.png) > > And I think I have them migrating here and there, and while they are > trying to find the new home for themselves, it's not possible to allocate > those 2 shards. > I've tried to close the index and then open - when the index was closed - > all the remaining shards were allocated successfully, after I've opened it > again - no shards remain unassigned. > > So the current question is - why some shards are relocating constantly. > > Regards, > > On Monday, November 17, 2014 11:02:43 AM UTC+2, Pavel P wrote: > > > Hi, > > > > I have a cluster from 5 nodes, where I store the information from the > > logstash. > > Recently I've tried to increase the number of shards in the logstash > > index to 20 (from 5). > > > > From the beginning everything went well, all the shards were allocated > > and the cluster state was green. > > > > But, currently, when the new index is started (at the beginning of the > > day), I met the situation when some shards are not allocated: > > > > [https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png](https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png) > > > > Nothing happens during the day, while I thing the cluster has resources. > > > > If I would restart some nodes from the cluster, it could turn out that > > all the shards would be allocated. > > > > The idea which I'm trying to reach - more shards =\> each shard is smaller > > =\> it would allocate them faster, it would index faster, because of the > > indexing on each shard. > > > > Questions: > > > > 1. Why it is not able to allocate those shards during the index creation? > > 2. Why it does not allocate those shards during the day? > > 3. What is the recommended shards number for N nodes? > > > > Any thoughts are welcome. > > > > Regards, -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3266fd9d-9b75-4670-a1d6-7380340cec4a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3266fd9d-9b75-4670-a1d6-7380340cec4a%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Jakub\_Podeszwik](https://avatars.discourse-cdn.com/v4/letter/j/a3d4f5/32.png) [@Jakub\_Podeszwik](https://discuss.elastic.co/u/Jakub_Podeszwik)\ **Post date:** [November 19, 2014, 7:24am UTC](https://discuss.elastic.co/t/20-shards-per-5-nodes-thoughts/20777/7 "2014-11-19T07:24:48Z")
I've seen similar behaviour when i've had checksum some errors.There may be some info in your logs about it. In that case shard fail to be assigned because elasticsearch couldn't verify checksum of one or more of its segments. On Tuesday, 18 November 2014 14:17:02 UTC+1, Pavel P wrote: > It looks, that I have 2 shards unassigned, because two other shards were > stuck in "RELOCATING" state. > > [https://lh5.googleusercontent.com/-TD84Uz8lg5Q/VGtGVXVIFVI/AAAAAAAAAK4/ITudefzX1qQ/s1600/ES\_RELOCATING.png](https://lh5.googleusercontent.com/-TD84Uz8lg5Q/VGtGVXVIFVI/AAAAAAAAAK4/ITudefzX1qQ/s1600/ES_RELOCATING.png) > > And I think I have them migrating here and there, and while they are > trying to find the new home for themselves, it's not possible to allocate > those 2 shards. > I've tried to close the index and then open - when the index was closed - > all the remaining shards were allocated successfully, after I've opened it > again - no shards remain unassigned. > > So the current question is - why some shards are relocating constantly. > > Regards, > > On Monday, November 17, 2014 11:02:43 AM UTC+2, Pavel P wrote: > > > Hi, > > > > I have a cluster from 5 nodes, where I store the information from the > > logstash. > > Recently I've tried to increase the number of shards in the logstash > > index to 20 (from 5). > > > > From the beginning everything went well, all the shards were allocated > > and the cluster state was green. > > > > But, currently, when the new index is started (at the beginning of the > > day), I met the situation when some shards are not allocated: > > > > [https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png](https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png) > > > > Nothing happens during the day, while I thing the cluster has resources. > > > > If I would restart some nodes from the cluster, it could turn out that > > all the shards would be allocated. > > > > The idea which I'm trying to reach - more shards =\> each shard is smaller > > =\> it would allocate them faster, it would index faster, because of the > > indexing on each shard. > > > > Questions: > > > > 1. Why it is not able to allocate those shards during the index creation? > > 2. Why it does not allocate those shards during the day? > > 3. What is the recommended shards number for N nodes? > > > > Any thoughts are welcome. > > > > Regards, -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c9ccce72-37ac-44d5-a016-ea8ef9a20e73%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c9ccce72-37ac-44d5-a016-ea8ef9a20e73%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Pavel\_P](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pavel_p/32/1092_2.png) [@Pavel\_P](https://discuss.elastic.co/u/Pavel_P)\ **Post date:** [November 19, 2014, 7:48am UTC](https://discuss.elastic.co/t/20-shards-per-5-nodes-thoughts/20777/8 "2014-11-19T07:48:35Z")
Thanks Jakub, it's interesting. For now I've solved the issue with the increasing the number of RELOCATING shards: [image: Inline image 1] Looks like it helps. Regards, On Wed, Nov 19, 2014 at 9:24 AM, Jakub Podeszwik [jakubpodeszwik@gmail.com](mailto:jakubpodeszwik@gmail.com) wrote: > I've seen similar behaviour when i've had checksum some errors.There may > be some info in your logs about it. In that case shard fail to be assigned > because elasticsearch couldn't verify checksum of one or more of its > segments. > > On Tuesday, 18 November 2014 14:17:02 UTC+1, Pavel P wrote: > > > It looks, that I have 2 shards unassigned, because two other shards were > > stuck in "RELOCATING" state. > > > > [https://lh5.googleusercontent.com/-TD84Uz8lg5Q/VGtGVXVIFVI/AAAAAAAAAK4/ITudefzX1qQ/s1600/ES\_RELOCATING.png](https://lh5.googleusercontent.com/-TD84Uz8lg5Q/VGtGVXVIFVI/AAAAAAAAAK4/ITudefzX1qQ/s1600/ES_RELOCATING.png) > > > > And I think I have them migrating here and there, and while they are > > trying to find the new home for themselves, it's not possible to allocate > > those 2 shards. > > I've tried to close the index and then open - when the index was closed - > > all the remaining shards were allocated successfully, after I've opened it > > again - no shards remain unassigned. > > > > So the current question is - why some shards are relocating constantly. > > > > Regards, > > > > On Monday, November 17, 2014 11:02:43 AM UTC+2, Pavel P wrote: > > > > > Hi, > > > > > > I have a cluster from 5 nodes, where I store the information from the > > > logstash. > > > Recently I've tried to increase the number of shards in the logstash > > > index to 20 (from 5). > > > > > > From the beginning everything went well, all the shards were allocated > > > and the cluster state was green. > > > > > > But, currently, when the new index is started (at the beginning of the > > > day), I met the situation when some shards are not allocated: > > > > > > [https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png](https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png) > > > > > > Nothing happens during the day, while I thing the cluster has resources. > > > > > > If I would restart some nodes from the cluster, it could turn out that > > > all the shards would be allocated. > > > > > > The idea which I'm trying to reach - more shards =\> each shard is > > > smaller =\> it would allocate them faster, it would index faster, because of > > > the indexing on each shard. > > > > > > Questions: > > > > > > 1. Why it is not able to allocate those shards during the index creation? > > > 2. Why it does not allocate those shards during the day? > > > 3. What is the recommended shards number for N nodes? > > > > > > Any thoughts are welcome. > > > > > > Regards, > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/q5Xkn9pc9EM/unsubscribe](https://groups.google.com/d/topic/elasticsearch/q5Xkn9pc9EM/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/c9ccce72-37ac-44d5-a016-ea8ef9a20e73%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c9ccce72-37ac-44d5-a016-ea8ef9a20e73%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/c9ccce72-37ac-44d5-a016-ea8ef9a20e73%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/c9ccce72-37ac-44d5-a016-ea8ef9a20e73%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- _Pavel Polyakov_ Software Engineer - PHP E-mail: [pavel@kredito.de](mailto:pavel@kredito.de) Skype: pavel.polyakov.x1 [https://www.facebook.com/kreditech](https://www.facebook.com/kreditech) Kreditech Holding SSL GmbH Am Sandtorkai 50, 20457 Hamburg, Germany Office phone: +49 (0)40 - 605905-60 Authorized representatives: Sebastian Diemer, Alexander Graubner-Müller Company registration: Hamburg HRB122027 [www.kreditech.com](http://www.kreditech.com) [Kreditech](http://facebook.com/kreditech) [https://www.facebook.com/kreditech](https://www.facebook.com/kreditech) [https://www.facebook.com/kreditech](https://www.facebook.com/kreditech) This e-mail contains confidential and/or legally protected information. If you are not the intended recipient or if you have received this e-mail by error please notify the sender immediately and destroy this e-mail. Any unauthorized review, copying, disclosure or distribution of the material in this e-mail is strictly forbidden. The contents of this e-mail is legally binding only if it is confirmed by letter or fax. The sending of e-mails to us does not have any period-protecting effect. Thank you for your cooperation. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAFVUaqO9zccHa-EO0e\_75\_B9ZNDuF\_RVCBkn9OLs0tK7\_fxnig%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAFVUaqO9zccHa-EO0e_75_B9ZNDuF_RVCBkn9OLs0tK7_fxnig%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [November 19, 2014, 10:35am UTC](https://discuss.elastic.co/t/20-shards-per-5-nodes-thoughts/20777/9 "2014-11-19T10:35:30Z")
Can you please not attach such large images to the list, it's easier for everyone if you can host and then link to them elsewhere instead 🙂 On 19 November 2014 18:48, Pavel P [pavel@kredito.de](mailto:pavel@kredito.de) wrote: > Thanks Jakub, it's interesting. > > For now I've solved the issue with the increasing the number of RELOCATING > shards: > [image: Inline image 1] > > Looks like it helps. > > Regards, > > On Wed, Nov 19, 2014 at 9:24 AM, Jakub Podeszwik \<[jakubpodeszwik@gmail.com](mailto:jakubpodeszwik@gmail.com) > > > wrote: > > > I've seen similar behaviour when i've had checksum some errors.There may > > be some info in your logs about it. In that case shard fail to be assigned > > because elasticsearch couldn't verify checksum of one or more of its > > segments. > > > > On Tuesday, 18 November 2014 14:17:02 UTC+1, Pavel P wrote: > > > > > It looks, that I have 2 shards unassigned, because two other shards were > > > stuck in "RELOCATING" state. > > > > > > [https://lh5.googleusercontent.com/-TD84Uz8lg5Q/VGtGVXVIFVI/AAAAAAAAAK4/ITudefzX1qQ/s1600/ES\_RELOCATING.png](https://lh5.googleusercontent.com/-TD84Uz8lg5Q/VGtGVXVIFVI/AAAAAAAAAK4/ITudefzX1qQ/s1600/ES_RELOCATING.png) > > > > > > And I think I have them migrating here and there, and while they are > > > trying to find the new home for themselves, it's not possible to allocate > > > those 2 shards. > > > I've tried to close the index and then open - when the index was closed > > > > > > - all the remaining shards were allocated successfully, after I've opened > > > it again - no shards remain unassigned. > > > > > > So the current question is - why some shards are relocating constantly. > > > > > > Regards, > > > > > > On Monday, November 17, 2014 11:02:43 AM UTC+2, Pavel P wrote: > > > > > > > Hi, > > > > > > > > I have a cluster from 5 nodes, where I store the information from the > > > > logstash. > > > > Recently I've tried to increase the number of shards in the logstash > > > > index to 20 (from 5). > > > > > > > > From the beginning everything went well, all the shards were allocated > > > > and the cluster state was green. > > > > > > > > But, currently, when the new index is started (at the beginning of the > > > > day), I met the situation when some shards are not allocated: > > > > > > > > [https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png](https://lh3.googleusercontent.com/-R3bWkXbXCbU/VGmwngnPOzI/AAAAAAAAAKY/9n8XlVCGPyc/s1600/Screen%2BShot%2B2014-11-17%2Bat%2B10.23.04%2BAM.png) > > > > > > > > Nothing happens during the day, while I thing the cluster has > > > > resources. > > > > > > > > If I would restart some nodes from the cluster, it could turn out that > > > > all the shards would be allocated. > > > > > > > > The idea which I'm trying to reach - more shards =\> each shard is > > > > smaller =\> it would allocate them faster, it would index faster, because of > > > > the indexing on each shard. > > > > > > > > Questions: > > > > > > > > 1. Why it is not able to allocate those shards during the index > > > > creation? > > > > 2. Why it does not allocate those shards during the day? > > > > 3. What is the recommended shards number for N nodes? > > > > > > > > Any thoughts are welcome. > > > > > > > > Regards, > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/q5Xkn9pc9EM/unsubscribe](https://groups.google.com/d/topic/elasticsearch/q5Xkn9pc9EM/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/c9ccce72-37ac-44d5-a016-ea8ef9a20e73%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c9ccce72-37ac-44d5-a016-ea8ef9a20e73%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/c9ccce72-37ac-44d5-a016-ea8ef9a20e73%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/c9ccce72-37ac-44d5-a016-ea8ef9a20e73%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > > _Pavel Polyakov_ > > Software Engineer - PHP > > E-mail: [pavel@kredito.de](mailto:pavel@kredito.de) > Skype: pavel.polyakov.x1 > > [https://www.facebook.com/kreditech](https://www.facebook.com/kreditech) > Kreditech Holding SSL GmbH > Am Sandtorkai 50, 20457 Hamburg, Germany > Office phone: +49 (0)40 - 605905-60 > Authorized representatives: Sebastian Diemer, Alexander Graubner-Müller > Company registration: Hamburg HRB122027 > > [www.kreditech.com](http://www.kreditech.com) > [Kreditech](http://facebook.com/kreditech) [https://www.facebook.com/kreditech](https://www.facebook.com/kreditech) > > [https://www.facebook.com/kreditech](https://www.facebook.com/kreditech) > > This e-mail contains confidential and/or legally protected information. If > you are not the intended recipient or if you have received this e-mail by > error please notify the sender immediately and destroy this e-mail. Any > unauthorized review, copying, disclosure or distribution of the material in > this e-mail is strictly forbidden. The contents of this e-mail is legally > binding only if it is confirmed by letter or fax. The sending of e-mails to > us does not have any period-protecting effect. Thank you for your > cooperation. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAFVUaqO9zccHa-EO0e\_75\_B9ZNDuF\_RVCBkn9OLs0tK7\_fxnig%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAFVUaqO9zccHa-EO0e_75_B9ZNDuF_RVCBkn9OLs0tK7_fxnig%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAFVUaqO9zccHa-EO0e\_75\_B9ZNDuF\_RVCBkn9OLs0tK7\_fxnig%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAFVUaqO9zccHa-EO0e_75_B9ZNDuF_RVCBkn9OLs0tK7_fxnig%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZ%3Dd1%3DhO\_mozsX%3D\_4%3DL%2B\_6CFwcpOvR%2BtbmxLQofyu-37Mw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZ%3Dd1%3DhO_mozsX%3D_4%3DL%2B_6CFwcpOvR%2BtbmxLQofyu-37Mw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:49am UTC](https://discuss.elastic.co/t/20-shards-per-5-nodes-thoughts/20777/10 "2017-07-06T00:49:01Z")