# 13mil documents and Elasticsearch status=red after server restart **URL:** \ **Category:** Elasticsearch\ **Created:** [April 9, 2014, 2:27pm UTC](https://discuss.elastic.co/t/13mil-documents-and-elasticsearch-status-red-after-server-restart/16900 "2014-04-09T14:27:24Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Madeleine](https://avatars.discourse-cdn.com/v4/letter/m/958977/32.png) [@Madeleine](https://discuss.elastic.co/u/Madeleine)\ **Post date:** [April 9, 2014, 2:27pm UTC](https://discuss.elastic.co/t/13mil-documents-and-elasticsearch-status-red-after-server-restart/16900/1 "2014-04-09T14:27:24Z")
Scenario: Log files parsed by logstash generated 13mil results in elasticsearch. After a server restart the shards are not allocated anymore. The Roy Russo dashboard says there's a problem with the swap space. I've tried setting mlockall to true, but then I found out that it only works on Linux and not Windows?... And it had no effect. I've found some conflicting information here[http://gibrown.wordpress.com/2013/12/05/managing-elasticsearch-cluster-restart-time/](http://gibrown.wordpress.com/2013/12/05/managing-elasticsearch-cluster-restart-time/)and here[http://stackoverflow.com/questions/19967472/elasticsearch-unassigned-shards-how-to-fix](http://stackoverflow.com/questions/19967472/elasticsearch-unassigned-shards-how-to-fix)regarding how to disable allocation (or at least it seams contradictory to me...) Any ideas? Thanks! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/11edbbd0-5305-4823-a854-114dfef410f4%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/11edbbd0-5305-4823-a854-114dfef410f4%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Madeleine](https://avatars.discourse-cdn.com/v4/letter/m/958977/32.png) [@Madeleine](https://discuss.elastic.co/u/Madeleine)\ **Post date:** [April 9, 2014, 3:57pm UTC](https://discuss.elastic.co/t/13mil-documents-and-elasticsearch-status-red-after-server-restart/16900/2 "2014-04-09T15:57:08Z")
I've also closed all indices, then started one only and waited... Still ended up with a red status and some unallocated shards. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/15a3da82-4730-415a-bf12-dd556d8b5177%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/15a3da82-4730-415a-bf12-dd556d8b5177%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [April 10, 2014, 11:27pm UTC](https://discuss.elastic.co/t/13mil-documents-and-elasticsearch-status-red-after-server-restart/16900/3 "2014-04-10T23:27:15Z")
How many nodes do you have? How many replicas? After a quick glance, it looks like both stack overflow answers have the correct way to disable allocation. -- Ivan On Wed, Apr 9, 2014 at 8:57 AM, Madeleine [mxmadalina@gmail.com](mailto:mxmadalina@gmail.com) wrote: > I've also closed all indices, then started one only and waited... Still > ended up with a red status and some unallocated shards. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/15a3da82-4730-415a-bf12-dd556d8b5177%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/15a3da82-4730-415a-bf12-dd556d8b5177%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/15a3da82-4730-415a-bf12-dd556d8b5177%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/15a3da82-4730-415a-bf12-dd556d8b5177%40googlegroups.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDm3a4%3DxqwvZXk%2B\_hnkDU5vojjaf-RNsO%2BjpswbnnnTVw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDm3a4%3DxqwvZXk%2B_hnkDU5vojjaf-RNsO%2BjpswbnnnTVw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 21, 2014, 11:44am UTC](https://discuss.elastic.co/t/13mil-documents-and-elasticsearch-status-red-after-server-restart/16900/4 "2014-04-21T11:44:40Z")
Hey, can you check the log files for error messages (possibly also before the restart)? Also the recovery API may help you do identify if you still recovering shards or recovery is finished and you had problems while recovering the data (again, checking the log files should help a lot) --Alex On Fri, Apr 11, 2014 at 1:27 AM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > How many nodes do you have? How many replicas? > > After a quick glance, it looks like both stack overflow answers have the > correct way to disable allocation. > > -- > Ivan > > On Wed, Apr 9, 2014 at 8:57 AM, Madeleine [mxmadalina@gmail.com](mailto:mxmadalina@gmail.com) wrote: > > > I've also closed all indices, then started one only and waited... Still > > ended up with a red status and some unallocated shards. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/15a3da82-4730-415a-bf12-dd556d8b5177%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/15a3da82-4730-415a-bf12-dd556d8b5177%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/15a3da82-4730-415a-bf12-dd556d8b5177%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/15a3da82-4730-415a-bf12-dd556d8b5177%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDm3a4%3DxqwvZXk%2B\_hnkDU5vojjaf-RNsO%2BjpswbnnnTVw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDm3a4%3DxqwvZXk%2B_hnkDU5vojjaf-RNsO%2BjpswbnnnTVw%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDm3a4%3DxqwvZXk%2B\_hnkDU5vojjaf-RNsO%2BjpswbnnnTVw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDm3a4%3DxqwvZXk%2B_hnkDU5vojjaf-RNsO%2BjpswbnnnTVw%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM\_AKG2ifzKEwH-aMcAVrDxSpnozAe-YKx9OCeQ9fG6WBA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_AKG2ifzKEwH-aMcAVrDxSpnozAe-YKx9OCeQ9fG6WBA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Madeleine](https://avatars.discourse-cdn.com/v4/letter/m/958977/32.png) [@Madeleine](https://discuss.elastic.co/u/Madeleine)\ **Post date:** [May 7, 2014, 10:42am UTC](https://discuss.elastic.co/t/13mil-documents-and-elasticsearch-status-red-after-server-restart/16900/5 "2014-05-07T10:42:18Z")
_Sense says_: { "count": 4640383, "\_shards": { "total": 115, "successful": 54, "failed": 0 } } _Roy Russo shows_: # Nodes 1 # Data Nodes 1 Active Primary Shards 54 Active Shards 54 Relocating Shards 0 Initializing Shards 0 Unassigned Shards 176 Problem is there is one node only shown by Roy Russo. There should be more... -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/03c1babd-62bd-4796-b14a-95b0f4c811f1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/03c1babd-62bd-4796-b14a-95b0f4c811f1%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Madeleine](https://avatars.discourse-cdn.com/v4/letter/m/958977/32.png) [@Madeleine](https://discuss.elastic.co/u/Madeleine)\ **Post date:** [May 7, 2014, 10:47am UTC](https://discuss.elastic.co/t/13mil-documents-and-elasticsearch-status-red-after-server-restart/16900/6 "2014-05-07T10:47:36Z")
There is no error in the logs. Just information like this: [2014-05-05 10:41:58,340][INFO][gateway] [Arides] recovered [40] indices into cluster\_state -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/fe34f92a-c945-469e-879b-c57ac7e493b9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/fe34f92a-c945-469e-879b-c57ac7e493b9%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 7, 2014, 11:05am UTC](https://discuss.elastic.co/t/13mil-documents-and-elasticsearch-status-red-after-server-restart/16900/7 "2014-05-07T11:05:50Z")
You need to check the other nodes then. Use a monitoring plugin as well, it's a lot easier. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 7 May 2014 20:47, Madeleine [mxmadalina@gmail.com](mailto:mxmadalina@gmail.com) wrote: > There is no error in the logs. Just information like this: > [2014-05-05 10:41:58,340][INFO][gateway] [Arides] > recovered [40] indices into cluster\_state > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/fe34f92a-c945-469e-879b-c57ac7e493b9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/fe34f92a-c945-469e-879b-c57ac7e493b9%40googlegroups.com)[https://groups.google.com/d/msgid/elasticsearch/fe34f92a-c945-469e-879b-c57ac7e493b9%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/fe34f92a-c945-469e-879b-c57ac7e493b9%40googlegroups.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624YpzhfLTMQ%2B7BovAzbjpVSrPmB6UzYm\_\_7fE5CSz9oc2w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YpzhfLTMQ%2B7BovAzbjpVSrPmB6UzYm__7fE5CSz9oc2w%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:31am UTC](https://discuss.elastic.co/t/13mil-documents-and-elasticsearch-status-red-after-server-restart/16900/8 "2017-07-06T01:31:06Z")