I am enrolled in/subscribed to the on-demand Elasticsearch Engineer course. The first four or five days were good, but a couple of days ago the lab, and maybe specifically cluster1 and/or kibana1, has started having some issues.
The problems started with kibana1 taking a long time to start, with the "not showing up?" and "Kibana server is not ready yet" messages. Waiting several minutes and clicking the refresh button at the lower right of the window would usually get kibana1 to the login prompt.
The problems progressed to queries (in the Dev Tools Console) failing with shard issues and (I didn't write down the error) "too much data" warnings. I could close and restart the lab and it would usually be good, but the problems became more frequent.
I now cannot get logged into kibana1 at all, and that's the interface to the cluster where all the important data for the labs is indexed - so my training progress has run aground. Kibana2 is fine, but 1 is stuck at "Kibana server is not ready yet." I have tried the "Restart workspace machines" to no effect. I can get into Terminal and ssh to kibana1, kibana2, server1, etc., but I don't think I have privileges to do much there (for instance, reboot requires root privs, and I don't seem to be able to elevate to that).
Are there any other troubleshooting or DIY fixes I can try? I sent a request to firstname.lastname@example.org and they responded with the usual "is it plugged in?" troubleshooting steps. I did those - the Strigo lab readiness checks all passed on both systems where I've been working (a laptop with Win10 and Chrome and a desktop with CentOS and Firefox). I've cleared cache, history, and cookies, and rebooted, with no change. Again, I've been working fine for several days, and some of the lab is accessible, so I'm thinking all necessary piecesparts are working on my end, and that part of my lab has broken, ceased functioning, is now floating at the top of the tank. Searching the posts here suggests I'm not the only one who's had this problem, but I'm not finding anything recent with a DIY fix. I've heard there's a lab restore script, but have not found formal documentation. I do see, in the Terminal, a docker/kibana directory with a build.sh script that looks like it might rebuild Kibana. At this point I'm willing to try it, so Florida Man is getting ready to say "hold m'beer and watch this!"