User Deleted All Files Inside Node0 Directory

Hey Folks,

So one if my users deleted everything inside my directory C:\ProgramData\Elastic\Elasticsearch\data\nodes\0\indices

'Now I can't start the service "elasticsearch" '

My worry is the configuration is totally bunked and I have to restart.
I'm fine with dataloss, that's not a problem. I just want my nodes up and running :frowning:

Any suggestions for what to do here?

Delete C:\ProgramData\Elastic\Elasticsearch\data\nodes\0. TBC this will lose absolutely everything from that node, but if you lost ...\indices then you're already a long way past the point of no return I'm afraid.

Will try that out and report findings. Honestly, the way we use it isn't for historical data. Data loss is not even remotely a worry.

edit looks like I still can't start the service.
image

Will edit once server restart is done.

Hmm, ok, there's something else going wrong then. There will be more detail in the server logs, can you share them here?

Winlogs_Cluster just looks like this

Nothing else seems to be logging in the programdata\elastic\elasticsearch\logs directory.

This is BEFORE the files in ..\node\0 got deleted. They got deleted because a user was trying to free up disk space incidental to Elastic dying (because of lack of disk space)

[2020-08-03T10:03:57,520][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:04:27,536][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:04:27,536][INFO ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2020-08-03T10:04:57,551][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:05:27,567][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:05:27,567][INFO ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2020-08-03T10:05:57,582][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:06:27,598][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:06:57,629][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:06:57,629][INFO ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2020-08-03T10:07:27,644][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:07:57,660][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:07:57,660][INFO ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2020-08-03T10:08:27,707][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:08:57,722][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:09:27,738][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:09:27,738][INFO ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2020-08-03T10:09:57,753][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:10:27,768][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:10:27,768][INFO ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2020-08-03T10:10:57,784][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:11:27,831][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:11:57,846][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:11:57,846][INFO ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2020-08-03T10:12:27,862][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:12:57,893][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:12:57,908][INFO ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2020-08-03T10:13:27,924][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:13:57,955][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:14:27,970][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:14:27,970][INFO ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2020-08-03T10:14:57,986][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:15:28,001][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:15:28,001][INFO ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2020-08-03T10:15:58,020][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:16:28,039][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:16:58,058][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:16:58,058][INFO ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2020-08-03T10:17:28,077][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:17:58,097][WARN ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] high disk watermark [90%] exceeded on [kLA52JfZTlCZr-dLG8mauQ][1SL-DEN-APM01][C:\ProgramData\Elastic\Elasticsearch\data\nodes\0] free: 13.8gb[5.5%], shards will be relocated away from this node
[2020-08-03T10:17:58,097][INFO ][o.e.c.r.a.DiskThresholdMonitor] [1SL-DEN-APM01] rerouting shards: [high disk watermark exceeded on one or more nodes]

Hmm that's not much help. It will be emitting information somewhere that says why it won't start up, but I don't know enough about Windows to tell you how to retrieve it, sorry.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.