# Unassigned Shards **URL:** \ **Category:** Elasticsearch\ **Created:** [December 19, 2013, 9:04pm UTC](https://discuss.elastic.co/t/unassigned-shards/14959 "2013-12-19T21:04:41Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Eric\_Luellen](https://avatars.discourse-cdn.com/v4/letter/e/ce73a5/32.png) [@Eric\_Luellen](https://discuss.elastic.co/u/Eric_Luellen)\ **Post date:** [December 19, 2013, 9:04pm UTC](https://discuss.elastic.co/t/unassigned-shards/14959/1 "2013-12-19T21:04:41Z")
Hello, Currently I have my syslog-ng --\> logstash --\> elasticsearch1 & elastisearch2 setup working pretty good. It's accepting over 300 events per second and hasn't bogged the systems down at all. However I'm running into 2 issues that I don't quite understand. 1. When viewing the information in Kibana, it appears to be anywhere from 15 min to an hr behind on the "all events" view. Sometimes when I search for new logs it shows up correctly but overall it seems like it's lagging behind trying to keep up with what logstash is sending it. That being said, I'm concerned that logs are being dropped and I don't know about it. Are there any commands I can use to validate this type of information or what I can do to make sure elasticsearch/KIbana is keeping up? 2. I've had to restart elasticsearch a few times and every time I do, it completely breaks things. Once it starts back up it doesn't continue to show the logs in Kibana correctly and when I run a health check, it says there are unassigned shards. I've not been able to fix this and in the past I've always just had to delete them and start from scratch again. Any idea what is going on with this or how I can more cleanly restart or reboot the servers and recover from it? Thanks, Eric -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/cab9c4e5-4e1a-4acd-b3ac-77fdc7ef6bef%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cab9c4e5-4e1a-4acd-b3ac-77fdc7ef6bef%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Eric\_Luellen](https://avatars.discourse-cdn.com/v4/letter/e/ce73a5/32.png) [@Eric\_Luellen](https://discuss.elastic.co/u/Eric_Luellen)\ **Post date:** [December 19, 2013, 9:52pm UTC](https://discuss.elastic.co/t/unassigned-shards/14959/2 "2013-12-19T21:52:18Z")
I think I made my situation even worse. I tried deleting the shards and starting over and now elasticsearch isn't even creating the /etc/elasticsearch/data/my-cluster/node folder. On Thursday, December 19, 2013 4:04:41 PM UTC-5, Eric Luellen wrote: > Hello, > > Currently I have my syslog-ng --\> logstash --\> elasticsearch1 & > elastisearch2 setup working pretty good. It's accepting over 300 events per > second and hasn't bogged the systems down at all. However I'm running into > 2 issues that I don't quite understand. > > 1. When viewing the information in Kibana, it appears to be anywhere from > 15 min to an hr behind on the "all events" view. Sometimes when I search > for new logs it shows up correctly but overall it seems like it's lagging > behind trying to keep up with what logstash is sending it. That being said, > I'm concerned that logs are being dropped and I don't know about it. Are > there any commands I can use to validate this type of information or what I > can do to make sure elasticsearch/KIbana is keeping up? > > 2. I've had to restart elasticsearch a few times and every time I do, it > completely breaks things. Once it starts back up it doesn't continue to > show the logs in Kibana correctly and when I run a health check, it says > there are unassigned shards. I've not been able to fix this and in the past > I've always just had to delete them and start from scratch again. > > Any idea what is going on with this or how I can more cleanly restart or > reboot the servers and recover from it? > > Thanks, > Eric -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1f9a1c4a-94cf-49d7-a4d1-22ffb0b64727%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1f9a1c4a-94cf-49d7-a4d1-22ffb0b64727%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 19, 2013, 10:07pm UTC](https://discuss.elastic.co/t/unassigned-shards/14959/3 "2013-12-19T22:07:53Z")
Did you install ES via a rpm/deb or using the zip? I ask because your data store directory is custom.subl Check out these plugins for monitoring - elastichq, kopf, bigdesk. They will give you an overview of your cluster and might give you insight into where your problem lies. The other best place to check is the ES logs. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 20 December 2013 08:52, Eric Luellen [eric.luellen@gmail.com](mailto:eric.luellen@gmail.com) wrote: > I think I made my situation even worse. I tried deleting the shards and > starting over and now elasticsearch isn't even creating the > /etc/elasticsearch/data/my-cluster/node folder. > > On Thursday, December 19, 2013 4:04:41 PM UTC-5, Eric Luellen wrote: > > > Hello, > > > > Currently I have my syslog-ng --\> logstash --\> elasticsearch1 & > > elastisearch2 setup working pretty good. It's accepting over 300 events per > > second and hasn't bogged the systems down at all. However I'm running into > > 2 issues that I don't quite understand. > > > > 1. When viewing the information in Kibana, it appears to be anywhere from > > 15 min to an hr behind on the "all events" view. Sometimes when I search > > for new logs it shows up correctly but overall it seems like it's lagging > > behind trying to keep up with what logstash is sending it. That being said, > > I'm concerned that logs are being dropped and I don't know about it. Are > > there any commands I can use to validate this type of information or what I > > can do to make sure elasticsearch/KIbana is keeping up? > > > > 2. I've had to restart elasticsearch a few times and every time I do, it > > completely breaks things. Once it starts back up it doesn't continue to > > show the logs in Kibana correctly and when I run a health check, it says > > there are unassigned shards. I've not been able to fix this and in the past > > I've always just had to delete them and start from scratch again. > > > > Any idea what is going on with this or how I can more cleanly restart or > > reboot the servers and recover from it? > > > > Thanks, > > Eric > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/1f9a1c4a-94cf-49d7-a4d1-22ffb0b64727%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1f9a1c4a-94cf-49d7-a4d1-22ffb0b64727%40googlegroups.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624YkhRdv1b8hA9kM%3D-f006Cxnu9zt5BJz1zMXAO%2BbdUEAw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624YkhRdv1b8hA9kM%3D-f006Cxnu9zt5BJz1zMXAO%2BbdUEAw%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Eric\_Luellen](https://avatars.discourse-cdn.com/v4/letter/e/ce73a5/32.png) [@Eric\_Luellen](https://discuss.elastic.co/u/Eric_Luellen)\ **Post date:** [December 20, 2013, 2:28pm UTC](https://discuss.elastic.co/t/unassigned-shards/14959/4 "2013-12-20T14:28:53Z")
Mark, I used the rpm install. I'll take a look at the plugins. Thanks. On Thursday, December 19, 2013 5:07:53 PM UTC-5, Mark Walkom wrote: > Did you install ES via a rpm/deb or using the zip? I ask because your data > store directory is custom.subl > > Check out these plugins for monitoring - elastichq, kopf, bigdesk. They > will give you an overview of your cluster and might give you insight into > where your problem lies. The other best place to check is the ES logs. > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \ > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > On 20 December 2013 08:52, Eric Luellen \<[eric.l...@gmail.com](mailto:eric.l...@gmail.com) \ > > > wrote: > > > I think I made my situation even worse. I tried deleting the shards and > > starting over and now elasticsearch isn't even creating the > > /etc/elasticsearch/data/my-cluster/node folder. > > > > On Thursday, December 19, 2013 4:04:41 PM UTC-5, Eric Luellen wrote: > > > > > Hello, > > > > > > Currently I have my syslog-ng --\> logstash --\> elasticsearch1 & > > > elastisearch2 setup working pretty good. It's accepting over 300 events per > > > second and hasn't bogged the systems down at all. However I'm running into > > > 2 issues that I don't quite understand. > > > > > > 1. When viewing the information in Kibana, it appears to be anywhere > > > from 15 min to an hr behind on the "all events" view. Sometimes when I > > > search for new logs it shows up correctly but overall it seems like it's > > > lagging behind trying to keep up with what logstash is sending it. That > > > being said, I'm concerned that logs are being dropped and I don't know > > > about it. Are there any commands I can use to validate this type of > > > information or what I can do to make sure elasticsearch/KIbana is keeping > > > up? > > > > > > 2. I've had to restart elasticsearch a few times and every time I do, it > > > completely breaks things. Once it starts back up it doesn't continue to > > > show the logs in Kibana correctly and when I run a health check, it says > > > there are unassigned shards. I've not been able to fix this and in the past > > > I've always just had to delete them and start from scratch again. > > > > > > Any idea what is going on with this or how I can more cleanly restart or > > > reboot the servers and recover from it? > > > > > > Thanks, > > > Eric > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/1f9a1c4a-94cf-49d7-a4d1-22ffb0b64727%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1f9a1c4a-94cf-49d7-a4d1-22ffb0b64727%40googlegroups.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/384e753f-6e9a-49d4-9db9-c03af81d9ba3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/384e753f-6e9a-49d4-9db9-c03af81d9ba3%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Eric\_Luellen](https://avatars.discourse-cdn.com/v4/letter/e/ce73a5/32.png) [@Eric\_Luellen](https://discuss.elastic.co/u/Eric_Luellen)\ **Post date:** [December 20, 2013, 8:05pm UTC](https://discuss.elastic.co/t/unassigned-shards/14959/5 "2013-12-20T20:05:40Z")
I got the initial issue fixed of me getting data back again. However I still don't understand how to fix the unassigned shards issue and how to properly restart elasticsearch without it complaining. On Friday, December 20, 2013 9:28:53 AM UTC-5, Eric Luellen wrote: > Mark, > > I used the rpm install. I'll take a look at the plugins. Thanks. > > On Thursday, December 19, 2013 5:07:53 PM UTC-5, Mark Walkom wrote: > > > Did you install ES via a rpm/deb or using the zip? I ask because your > > data store directory is custom.subl > > > > Check out these plugins for monitoring - elastichq, kopf, bigdesk. They > > will give you an overview of your cluster and might give you insight into > > where your problem lies. The other best place to check is the ES logs. > > > > Regards, > > Mark Walkom > > > > Infrastructure Engineer > > Campaign Monitor > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > On 20 December 2013 08:52, Eric Luellen [eric.l...@gmail.com](mailto:eric.l...@gmail.com) wrote: > > > > > I think I made my situation even worse. I tried deleting the shards and > > > starting over and now elasticsearch isn't even creating the > > > /etc/elasticsearch/data/my-cluster/node folder. > > > > > > On Thursday, December 19, 2013 4:04:41 PM UTC-5, Eric Luellen wrote: > > > > > > > Hello, > > > > > > > > Currently I have my syslog-ng --\> logstash --\> elasticsearch1 & > > > > elastisearch2 setup working pretty good. It's accepting over 300 events per > > > > second and hasn't bogged the systems down at all. However I'm running into > > > > 2 issues that I don't quite understand. > > > > > > > > 1. When viewing the information in Kibana, it appears to be anywhere > > > > from 15 min to an hr behind on the "all events" view. Sometimes when I > > > > search for new logs it shows up correctly but overall it seems like it's > > > > lagging behind trying to keep up with what logstash is sending it. That > > > > being said, I'm concerned that logs are being dropped and I don't know > > > > about it. Are there any commands I can use to validate this type of > > > > information or what I can do to make sure elasticsearch/KIbana is keeping > > > > up? > > > > > > > > 2. I've had to restart elasticsearch a few times and every time I do, > > > > it completely breaks things. Once it starts back up it doesn't continue to > > > > show the logs in Kibana correctly and when I run a health check, it says > > > > there are unassigned shards. I've not been able to fix this and in the past > > > > I've always just had to delete them and start from scratch again. > > > > > > > > Any idea what is going on with this or how I can more cleanly restart > > > > or reboot the servers and recover from it? > > > > > > > > Thanks, > > > > Eric > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/1f9a1c4a-94cf-49d7-a4d1-22ffb0b64727%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1f9a1c4a-94cf-49d7-a4d1-22ffb0b64727%40googlegroups.com) > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9c2df0af-0f11-485d-b892-ca8d7b8a108f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9c2df0af-0f11-485d-b892-ca8d7b8a108f%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:59am UTC](https://discuss.elastic.co/t/unassigned-shards/14959/6 "2017-07-06T01:59:53Z")